Express.js on Lambda Getting Started

AWS Lambda is a cost efficient and easy way to deploy server applications. Express.js is a very popular Node.js framework that makes it very easy to develop REST APIs. This post will go through the basics of deploying an Express.js application to AWS Lambda.

You can also check out the sample project on GitHub.

Develop Express.js Server

We first need to implement our Express.js server. Nothing particular we need to keep in mind here. We can simply define routes etc. as we normally would:

import express from 'express';
import cors from 'cors';
import helmet from 'helmet';
import { rootHandler } from './root';
export const app: express.Application = express();
app.use(helmet());
if (process.env.CORS) {
console.info(`Starting server with CORS domain: ${process.env.CORS}`);
app.use(cors({ origin: process.env.CORS, credentials: true }));
}
app.use(express.json());
app.get('/', rootHandler);
view raw server.ts hosted with ❤ by GitHub

In order to publish this server in a Lambda, we will need to add the aws-serverless-express to our project. Once that is done, we can define a new file lambda.ts with the following content:

require('source-map-support').install();
import awsServerlessExpress from 'aws-serverless-express';
import { app } from './server';
const server = awsServerlessExpress.createServer(app);
exports.handler = (event: any, context: any): any => {
awsServerlessExpress.proxy(server, event, context);
};
view raw lambda.ts hosted with ❤ by GitHub

Note that we are importing the app object from our server.ts file. We have also added the package source-map-support. Initialising this module in our code will result in much easier to read stack traces in the Lambda console (since we will package up our Lambda with Webpack).

Please see all files that are required for the server, including the handler in the sample project.

Package Server

In order to deploy our server to AWS lambda, we need to package it up into a ZIP file. Generally lambda accepts any Node.js application definition in the ZIP file but we will package up our application using Webpack. This will drastically reduce the size of our server, which results in much improved cold start times for our Lambda.

For this, we simply add the webpack package to our project and define a webpack.config.js as follows:

/* eslint-disable @typescript-eslint/no-var-requires */
const path = require('path');
const PnpWebpackPlugin = require('pnp-webpack-plugin');
module.exports = {
entry: './dist/src/lambda.js',
output: {
path: path.resolve(__dirname, 'distLambda'),
filename: 'lambda.js',
libraryTarget: 'umd',
},
target: 'node',
mode: 'production',
devtool: 'source-map',
resolve: {
plugins: [PnpWebpackPlugin],
},
resolveLoader: {
plugins: [PnpWebpackPlugin.moduleLoader(module)],
},
module: {
rules: [
// this is required to load source maps of libraries
{
test: /\.(js|js\.map|map)$/,
enforce: 'pre',
use: [require.resolve('source-map-loader')],
},
],
},
};
view raw webpack.config.js hosted with ❤ by GitHub

Note that we are adding some configuration here to include source maps for easy to read stack traces, as well as load an additional plugin to support Yarn Pnp which is used in the sample project.

Running webpack should result in the following files being generated:

lambda.js should be around 650 kb which includes the whole Express server plus Helmet which we included earlier. Cold-starts for lambda should be sub 1 s with this file size.

Deploy to AWS

Lastly, we need to deploy this Lambda to AWS. For this we first need to define the infrastructure for the Lambda. In the sample project, this is done in Terraform:

resource "aws_lambda_function" "main" {
function_name = var.lambda_name
filename = data.archive_file.empty_lambda.output_path
handler = "lambda.handler"
runtime = "nodejs12.x"
memory_size = 2048
timeout = 900
role = aws_iam_role.lambda_exec.arn
lifecycle {
ignore_changes = [
filename,
]
}
}
view raw lambda.tf hosted with ❤ by GitHub

Important here is handler which should match the file name and main method name for our packaged Node.js application. The filename is lambda.js and we defined an export handler in lambda.ts above. Therefore the handler we need to define for the Lambda is lambda.handler. Also note we set the runtime to nodejs12.x. This ensures that Lambda knows to run our application as a Node.js application. To see how to define a Lambda function manually, see this post.

Note that there is a bit more we need to configure, including an API gateway that will send through HTTP requests to our Lambda. To see all infrastructure definitions, see the AWS Infrastructure definitions in the sample project. One important thing to note is that we need to use a proxy integration in our API gateway. This will ensure that our lambda receives all HTTP calls for the gateway and allow our Express server to do the routing.

resource "aws_api_gateway_integration" "lambda" {
rest_api_id = aws_api_gateway_rest_api.main.id
resource_id = aws_api_gateway_method.proxy.resource_id
http_method = aws_api_gateway_method.proxy.http_method
# Lambdas can only be invoked via Post – but the gateway will also forward GET requests etc.
integration_http_method = "POST"
type = "AWS_PROXY"
uri = aws_lambda_function.main.invoke_arn
}
view raw api-gateway.tf hosted with ❤ by GitHub

Once our Lambda is created, we can simply ZIP up the folder distLambda/ and upload this to AWS using the Lambda console.

In the supplied sample project, we use @goldstack/template-lambda-express to help us with uploading our Lambda. Under the hood, this is using the AWS CLI. You can also use the AWS CLI directly using the update-function-code operation.

Next Steps

This post described a few fundamentals about deploying an AWS Lambda with an Express server. There are actually quite a number of steps involved to get a project working end to end. The easiest way I would recommend for getting a project with an Express server deployed on Lambda off the ground would be to use the tool Goldstack that I have developed, specifically the Express Lambda template. This has also been used to create the sample project for this post.

Otherwise, be welcome to check out the sample project on GitHub and modify it to your need. Note that one thing you will need to do is to update the goldstack.json configuration in packages/lambda-express/goldstack.json. Specifically you will need to change the domain configuration.

{
"$schema": "./schemas/package.schema.json",
"name": "lambda-express",
"template": "lambda-express",
"templateVersion": "0.1.19",
"configuration": {},
"deployments": [
{
"name": "prod",
"configuration": {
"lambdaName": "expressjs-lambda-getting-started",
"apiDomain": "expressjs-lambda.examples.goldstack.party",
"hostedZoneDomain": "goldstack.party"
},
"awsUser": "awsUser",
"awsRegion": "us-west-2"
}
]
}
view raw goldstack.json hosted with ❤ by GitHub

More details about the properties in this configuration can be found here

You will also need to create a config.json in config/infra/aws/config.json with AWS credentials for creating the infrastructure and deploying the Lambda.

{
"users": [
{
"name": "awsUser",
"type": "apiKey",
"config": {
"awsAccessKeyId": "your secret",
"awsSecretAccessKey": "your access key",
"awsDefaultRegion": "us-west-2"
}
}
]
}
view raw config.json hosted with ❤ by GitHub

If you simply use the Goldstack UI to configure your project, all these files will be prepared for you, and you can also easily create a fully configured monorepo that also includes modules for a React application or email sending.

Textures and Lighting with React and Three.js

In my previous three posts, I have developed a simple WebGL application using react-three-fiber and three.js. In this post, I am adding texture loading and proper lighting to the application.

For reference, here the links to the previous versions of the app:

  • Version 1: Just being able to drag a shape on the screen
  • Version 2: Dragging and dropping shapes using physics
  • Version 3: Being able to move the camera

Here the version developed for this post:

threejs-react-textures-light

Source Code

You can click to add objects, click and drag them as well as move the camera using WASD keys and mouse wheel to zoom.

Loading Textures

Textures can be loaded easily in react-three-fiber using the useResource hook.

All that is required to place the texture in the public/ folder of the react application, load the texture and then link it to the material by setting the map property.

    const [texture] = useLoader(TextureLoader, 'textures/grasslight-big.jpg');

    if (texture) {
        texture.wrapS = texture.wrapT = THREE.RepeatWrapping;
        texture.repeat.set(1500, 1500);
        texture.anisotropy = 16;
    }

    return (
        <mesh ref={ref} receiveShadow position={position}
            onClick={onPlaneClick}>
            <planeBufferGeometry attach="geometry" args={[10000, 10000]} />
            {texture &&
                <meshPhongMaterial attach="material" map={texture} />
            }

        </mesh>
    )

I found that textures are often quite large in size; larger than 1 MB. This significantly extends loading times. Thus I have added a simple loading screen. Unfortunately to be able to display the text ‘loading’ I had to create a TextGeometry which in turn required a font to be loaded (I prepared the Roboto font using facetype.js. This font by itself is more than 300 kb, so even loading the loading screen takes a bit of time.

Lighting

The goal of this application is to have a simple, very large plane on which any number of objects may be added. The issue I encountered with this was that to get shadows working with a DirectionalLight turned out to be very tricky. In the end, I used a combination of an AmbientLight with a SpotLight.

        <ambientLight intensity={0.9} />

        <primitive object={lightTarget} position={lightTargetPosition} />
        <spotLight
            castShadow
            intensity={0.25}
            position={lightPosition}
            angle={Math.PI / 3}
            penumbra={1}
            shadow-mapSize={new Vector2(2048 * 5, 2048 * 5)}
            target={lightTarget}
        />

Since the SpotLight would not be able to cover the whole of the plane (as said, it is meant to be very large) and provide accurate shadows, I opted for moving the SpotLight when a user moves the camera.

    const lightTargetYDelta = 120;
    const lightTargetXDelta = 80;
    const [lightPosition, setLightPosition] = useState([-lightTargetXDelta, -lightTargetYDelta, 200]);
    const [lightTargetPosition, setLightTargetPosition] = useState([0, 0, 0]);
    const onCameraMoved = (delta) => {
        const newLightPosition = delta.map((e, idx) => lightPosition[idx] + e);
        setLightPosition(newLightPosition);
        const newLightTargetPosition = [newLightPosition[0] + lightTargetXDelta, newLightPosition[1] + lightTargetYDelta, 0];
        setLightTargetPosition(newLightTargetPosition);
    };

This required both updating the position of the light setLightPosition as well as moving the light target setLightTargetPosition.

Modularity

Since the amount of code for this example increased quite a bit over the past three iterations, I broke up the application into multiple modules, with most React components now sitting in their own file.

I think this really shows the advantage of using React with Three.js, since it is easy for each component to manage its own state.

For the next iteration, I will most likely be looking at how I can remove the textures or use much smaller textures. I would like the application to be able to load as quickly as possible, and textures clearly do not seem a great option for this.

Camera Movement with Three.js

I have recently been working on small example application using three.js and react-three-fiber. In the first two iterations, I first developed a simple draggable shape floating in space and then supported multiple shapes that can be moved on a physical plane. In this post, I am going to be extending the example to support camera movements. Here links to the previous two iterations and the one developed in this post:

Prototypes

Iteration 3: Camera Movements (this post)

Source Code: threejs-test

Published App: three-js-camera.surge.sh

You can move the camera using WASD and zoom in and out using the mouse scroll wheel. You can create new objects by clicking on any empty spot on the plane.

Iteration 2: Movable objects on Plane

Blog Post: Create and Drag Shapes with Three.js, React and Cannon.js

Published App: react-three-fiber-draggable-v2

Iteration 1: Draggable Shape in Space

Blog Post: Creating a Draggable Shape with React Three Fiber

Published App: react-three-fiber-draggable.surge.sh

In the following I will describe the two ways in which camera movement is supported:

Camera Movements

Using the Keyboard

The easiest way to move the camera using the keyboard would be to change the position of the camera on key press. That is what I first tried and it turned out to be a very insufficient solution. Instead, I stored for how long each key is pressed in an object and then calculate camera movement for every frame.

const keyPressed = {
}

function App() {
    ...
    const handleKeyDown = (e) => {
        if (!keyPressed[e.key]) {
            keyPressed[e.key] = new Date().getTime();
        }
    };

    const handleKeyUp = (e) => {
        delete keyPressed[e.key];
    };
    ...
}

In react-three-fiber, the useful useFrame hook is available where we then calculate the camera movement:

    useFrame((_, delta) => {
        // move camera according to key pressed
        Object.entries(keyPressed).forEach((e) => {
            const [key, start] = e;
            const duration = new Date().getTime() - start;

            // increase momentum if key pressed longer
            let momentum = Math.sqrt(duration + 200) * 0.01 + 0.05;

            // adjust for actual time passed
            momentum = momentum * delta / 0.016;

            // increase momentum if camera higher
            momentum = momentum + camera.position.z * 0.02;

            switch (key) {
                case 'w': camera.translateY(momentum); break;
                case 's': camera.translateY(-momentum); break;
                case 'd': camera.translateX(momentum); break;
                case 'a': camera.translateX(-momentum); break;
                default:
            }
        });
    });

We first calculate the duration how long a key is pressed and then use this to determine the momentum. Finally we use this momentum to update the position of the camera.

Using the Mouse Wheel

We use the mouse wheel to zoom in and out. For this, the position of the camera needs to change along the z axis.

    const mouseWheel = (e) => {
        let delta = e.wheelDelta;
        delta = delta / 240;
        delta = -delta;
        if (delta <= 0) {
            delta -= camera.position.z * 0.1;
        } else {
            delta += camera.position.z * 0.1;
        }
        if (camera.position.z + delta > 1 && camera.position.z + delta < 200) {
            camera.translateZ(delta);
        }
    };

Here we simply determine the direction in which the wheel is scrolled and adjust the position of the camera to be higher or lower accordingly. There is a certain range in which the camera is permitted. If it gets to close to the ground or too far away from it, its position is not changed anymore even if the mouse wheel is turned.

I further had to change the way the drag movement was implemented. This originally worked only using the position of the mouse on the screen, since the screen never moved.

In case of an dynamic camera, we need a bit more calculation to be done which I encapsulated in the get3DPosition method.

const get3DPosition = ({ screenX, screenY, camera }) => {
    var vector = new THREE.Vector3(screenX, screenY, 0.5);
    vector.unproject(camera);
    var dir = vector.sub(camera.position).normalize();
    var distance = - camera.position.z / dir.z;
    var pos = camera.position.clone().add(dir.multiplyScalar(distance));
    return [pos.x, pos.y, 0];
};

The major limitation I want to work on next is the lightning. There is currently a SpotLight light source that only covers a small part of the plane and objects which are not within the cone of this spotlight are not rendered in an aesthetically pleasing fashion.

The full source code is available on GitHub.

Create and Drag Shapes with Three.js, React and Cannon.js

Following up from my article published a few days ago, I have now extended and improved the simple WebGL application that I originally developed using Three.js and react-three-fiber.

Version 1 of the application allowed dragging a simple shape around on the screen:

App: https://react-three-fiber-draggable.surge.sh/

Source Code: https://github.com/mxro/threejs-test/tree/master/test1

Version 2 combines this basic premise with the cannon.js physics engine. Multiple objects can now be created and they drop down onto a solid plane, on which they can be moved.

App: https://react-three-fiber-draggable-v2.surge.sh/

Source Code: https://github.com/mxro/threejs-test/tree/master/test2

Simply click the canvas to add new shapes that then can be dragged around the plane.

The most important logic for this solution is in the DraggableDodecahedron component:

function DraggableDodecahedron({ position: initialPosition }) {
    const { size, viewport } = useThree();
    const [position, setPosition] = useState(initialPosition);
    const [quaternion, setQuaternion] = useState([0, 0, 0, 0]);
    const aspect = size.width / viewport.width;

    const { ref, body } = useCannon({ bodyProps: { mass: 100000 } }, body => {
        body.addShape(new CANNON.Box(new CANNON.Vec3(1, 1, 1)))
        body.position.set(...position);
    }, []);

    const bind = useDrag(({ offset: [,], xy: [x, y], first, last }) => {
        if (first) {
            body.mass = 0;
            body.updateMassProperties();
        } else if (last) {
            body.mass = 10000;
            body.updateMassProperties();
        }
        body.position.set((x - size.width / 2) / aspect, -(y - size.height / 2) / aspect, -0.7);
    }, { pointerEvents: true });

    useFrame(() => {
        // Sync cannon body position with three js
        const deltaX = Math.abs(body.position.x - position[0]);
        const deltaY = Math.abs(body.position.y - position[1]);
        const deltaZ = Math.abs(body.position.z - position[2]);
        if (deltaX > 0.001 || deltaY > 0.001 || deltaZ > 0.001) {
            setPosition(body.position.clone().toArray());
        }
        const bodyQuaternion = body.quaternion.toArray();
        const quaternionDelta = bodyQuaternion.map((n, idx) => Math.abs(n - quaternion[idx]))
            .reduce((acc, curr) => acc + curr);
        if (quaternionDelta > 0.01) {
            setQuaternion(body.quaternion.toArray());
        }
    });
    return (
        <mesh ref={ref} castShadow position={position} quaternion={quaternion} {...bind()}
            onClick={e => {
                e.stopPropagation();
            }}
        >

            <dodecahedronBufferGeometry attach="geometry" />
            <meshLambertMaterial attach="material" color="yellow" />

        </mesh>
    )
}

Most notably here are three React hooks:

With the first hook we create a Cannon body that is set to the same dimension and position as the shape.

     const { ref, body } = useCannon({ bodyProps: { mass: 100000 } }, body => {
        body.addShape(new CANNON.Box(new CANNON.Vec3(1, 1, 1)))
        body.position.set(...position);
    }, []);

In the second hook, we use react-use-gesture to react to drag events. We temporarily set the mass of the body/shape to be moved to 0 and reset it to the original mass once the drag gesture is complete. Finally we also set the position of the cannon.js body to the position that the drag gesture current indicates.

      const bind = useDrag(({ offset: [,], xy: [x, y], first, last }) => {
        if (first) {
            body.mass = 0;
            body.updateMassProperties();
        } else if (last) {
            body.mass = 10000;
            body.updateMassProperties();
        }
        body.position.set((x - size.width / 2) / aspect, -(y - size.height / 2) / aspect, -0.7);
    }, { pointerEvents: true });

The third hook, useFrame(), runs a callback function before every frame is rendered (this hook is supplied by react-three-fiber). This is used here to synchronise the position of the body in cannon.js with the three.js shape. Since cannon.js updates positions with a very fine granularity, we first assert that a body has changed its position or orientation by a significant margin. Only if this is the case, we update the shape. This helps React to avoid unnecessary updates of the ‘virtual dom’.

    useFrame(() => {
        // Sync cannon body position with three js
        const deltaX = Math.abs(body.position.x - position[0]);
        const deltaY = Math.abs(body.position.y - position[1]);
        const deltaZ = Math.abs(body.position.z - position[2]);
        if (deltaX > 0.001 || deltaY > 0.001 || deltaZ > 0.001) {
            setPosition(body.position.clone().toArray());
        }
        const bodyQuaternion = body.quaternion.toArray();
        const quaternionDelta = bodyQuaternion.map((n, idx) => Math.abs(n - quaternion[idx]))
            .reduce((acc, curr) => acc + curr);
        if (quaternionDelta > 0.01) {
            setQuaternion(body.quaternion.toArray());
        }
    });

Apart from these hooks there is a simple click handler that stops click events from propagating. This is to prevent the event handler defined for the plane to trigger (this triggers adding a new shape to the scene).

Next I will be adding camera movement to this example.

Creating a Draggable Shape with React Three Fiber

I recently became interested how to render 3D graphics in the browser. I think WebGL is an extremely powerful technology and may one day become an important way of rendering content on the web.

There are various frameworks and tools available to use WebGL such as Babylon.js and three.js. To me, three.js looks the most promising for the use cases I am interested in.

For simple examples, three.js works beautifully but I think more complex applications can easily become unwieldy when using this framework. Thus I was very happy to come across react-three-fiber, which provides a wrapper around three.js using React. React, for all its shortcomings, is a powerful way to keep code modular and maintainable.

To get my hands dirty with this library, I have created a little example of an application that renders a Dodecahedron and allows dragging this shape by tapping or clicking and dragging with the mouse.

Here the link to the deployed application:

react-three-fiber-draggable.surge.sh

And here the link to the source code:

github.com/mxro/threejs-test/tree/master/test1

I think the source code is pretty self-explanatory. Essentially all logic is encapsulated into index.js:

import ReactDOM from "react-dom"
import React, { useRef, useState } from "react"
import { Canvas, useThree, useFrame } from "react-three-fiber"
import { useDrag } from "react-use-gesture"
import "./index.css"

function DraggableDodecahedron() {
    const colors = ["hotpink", "red", "blue", "green", "yellow"];
    const ref = useRef();
    const [colorIdx, setColorIdx] = useState(0);
    const [position, setPosition] = useState([0, 0, 0]);
    const { size, viewport } = useThree();
    const aspect = size.width / viewport.width;
    useFrame(() => {
        ref.current.rotation.z += 0.01
        ref.current.rotation.x += 0.01
    });
    const bind = useDrag(({ offset: [x, y] }) => {
        const [,, z] = position;
        setPosition([x / aspect, -y / aspect, z]);
    }, { pointerEvents: true });

    return (
        <mesh position={position} {...bind()}
            ref={ref}
            onClick={e => {
                if (colorIdx === 4) {
                    setColorIdx(0);
                } else {
                    setColorIdx(colorIdx+1);
                }
            }}
            onPointerOver={e => console.log('hover')}
            onPointerOut={e => console.log('unhover')}>

            <dodecahedronBufferGeometry attach="geometry" />
            <meshLambertMaterial attach="material" color={colors[colorIdx]} />

        </mesh>
    )
}

ReactDOM.render(
    <Canvas>
        <spotLight intensity={1.2} position={[30, 30, 50]} angle={0.2} penumbra={1} castShadow />
        <DraggableDodecahedron />
    </Canvas>,
    document.getElementById("root")
)

Noteworthy here is that instead of creating a Material, Geometry and Mesh directly, they are defined in JSX. Also, instead of having to request an animation frame, we are using the hook useFrame to drive the animation for our component.

I think it can easily be seen how react-three-fiber could be used to make three.js applications more modular, for instance by handling the animation specifically for each component. I think this project is also testament to the power of React in that it cannot only be used with the DOM but also with other rendering technologies.

Testing Apollo Client/Server Applications

Following up on the GraphQL, Node.JS and React Monorepo Starter Kit and GraphQL Apollo Starter Kit (Lerna, Node.js), I have now created an extended example which includes facilities to run unit and integration tests using Jest.

The code can be found on GitHub:

apollo-client-server-tests

The following tests are included:

React Component Test

This tests asserts a react component is rendered correctly. Backend data from GraphQL is supplied via a mock [packages/client-components/src/Books/Books.test.js]

import React from 'react';
import Books from './Books';

import renderer from 'react-test-renderer'
import { MockedProvider } from 'react-apollo/test-utils';

import GET_BOOK_TITLES from './graphql/queries/booktitles';

import wait from 'waait';

const mocks = [
  {
    request: {
      query: GET_BOOK_TITLES
    },
    result: {
      data: {
        books: [
          {
            title: 'Harry Potter and the Chamber of Secrets',
            author: 'J.K. Rowling',
          },
          {
            title: 'Jurassic Park',
            author: 'Michael Crichton',
          }
        ]
      },
    },
  },
];

it('Renders one book', async () => {

  const component = renderer.create(<MockedProvider mocks={mocks} addTypename={false}>
    <Books />
  </MockedProvider>);
  expect(component.toJSON()).toEqual('Loading...');

  // to wait for event loop to complete - after which component should be loaded
  await wait(0);

  const pre = component.root.findByType('pre');
  expect(pre.children).toContain('Harry Potter and the Chamber of Secrets');

});

GraphQL Schema Test

Based on the article Extensive GraphQL Testing in 3 minutes, this test verifies the GraphQL schema is defined correctly for running the relevant queries [packages/server-books/src/schema/index.test.js].

import {
    makeExecutableSchema,
    addMockFunctionsToSchema,
    mockServer
} from 'graphql-tools';

import { graphql } from 'graphql';

import booksSchema from './index';

const titleTestCase = {
    id: 'Query Title',
    query: `
      query {
        books {
            title
        }
      }
    `,
    variables: {},
    context: {},
    expected: { data: { books: [{ title: 'Title'} , { title: 'Title' }] } }
};

const cases = [titleTestCase];

describe('Schema', () => {
    const typeDefs = booksSchema;
    const mockSchema = makeExecutableSchema({ typeDefs });

    addMockFunctionsToSchema({
        schema: mockSchema,
        mocks: {
            Boolean: () => false,
            ID: () => '1',
            Int: () => 1,
            Float: () => 1.1,
            String: () => 'Title',
        }
    });

    test('Has valid type definitions', async () => {
        expect(async () => {
            const MockServer = mockServer(typeDefs);

            await MockServer.query(`{ __schema { types { name } } }`);
        }).not.toThrow();
    });

    cases.forEach(obj => {
        const { id, query, variables, context: ctx, expected } = obj;

        test(`Testing Query: ${id}`, async () => {
            return await expect(
                graphql(mockSchema, query, null, { ctx }, variables)
            ).resolves.toEqual(expected);
        });
    });

});

GraphQL Schema and Resolver Test

Extending the previous test as suggested by the article Effective Testing a GraphQL Server, this test affirms that GraphQL schema and resolvers are working correctly [packages/server-books/tests/Books.test.js].

import { makeExecutableSchema } from 'graphql-tools'
import { graphql } from 'graphql'
import resolvers from '../src/resolvers'
import typeDefs from '../src/schema'

const titleTestCase = {
    id: 'Query Title',
    query: `
      query {
        books {
            title
        }
      }
    `,
    variables: {},
    context: {},
    expected: { data: { books: [{ title: 'Harry Potter and the Chamber of Secrets' }, { title: 'Jurassic Park' }] } }
};

describe('Test Cases', () => {

    const cases = [titleTestCase]
    const schema = makeExecutableSchema({ typeDefs: typeDefs, resolvers: { Query: resolvers } })

    cases.forEach(obj => {
        const { id, query, variables, context, expected } = obj

        test(`query: ${id}`, async () => {
            const result = await graphql(schema, query, null, context, variables)
            return expect(result).toEqual(expected)
        })
    })
})

Conclusion

As with the previous two articles on getting started with Apollo etc. the code developed again aims to be as minimalistic as possible. It shows how Apollo client/server code may be tested in three different ways. These are quite exhaustive, even if the presented tests are simplistic.

The only test missing is an integration tests which will test the React component linked to a live Apollo server back-end. I am not sure if it is possible to run an ’embedded’ Apollo server in the browser. Running such a server for testing the React component would also be a good addition.

GraphQL Apollo Starter Kit (Lerna, Node.js)

In many ways developing in Node.js is very fast and lightweight. However it can also be bewildering at times. Mostly so since for everything there seems to be more than one established way of doing things. Moreover, the right way to do something can change within 3 to 6 months, so best practices documented in books and blogs quickly become obsolete.

Some days ago I wanted to create a simple sample project that shows how to develop a simple client/server application using Apollo and React. I thought this would be a simple undertaking. I imagined I would begin with a simple tutorial or ‘starter kit’ and quickly be up and running.

I found that things were not as easy as I imagined, since there are plenty of examples to spin up a simple react app chiefly using create-react-app. I also found the awesome apollo-universal-starter-kit. However, the former lacks insight of how to link this to a Node.js back-end server and the latter is a bit complex and opinionated in some ways as to which frameworks to use.

This motivated me to develop a small, simple ‘starter kit’ for Apollo Client and Server in GraphQL with the following features:

  • Uses ES 2018 ‘vanilla’ JavaScript
  • Uses Lerna to define project with two packages (client and server)
  • Uses Node.js / Express as web server
  • Uses Babel to allow using ES 6 style modules for Node.js
  • Provides easy ways to start a development server and spin up a production instance

Here a link to the project on GitHub:

graphql-apollo-starter-kit

The README provides the instructions to run and build the project.

A few things of note about the implementation:

This project is aimed at developers new to Node.js/React/Apollo. It is not implemented in the most elegant way possible but in a way which makes it easy to understand what is going on by browsing through the code. Be welcome to check out the project and modify it!

Free Tool for Renaming Files with Current Date

For many files, it is good to keep track of when they were originally created. In theory, every file created on any mainstream operating system should have its ‘creation date’ and ‘last date modified’ recorded as part of its metadata. This mechanisms, however, is often unreliable, especially when using file synchronisation tools such as Dropbox or Google Drive.

Thus it is often handy to prefix file names with the date they were created, such as:

2018 09 30 Letter.pdf

There are some nice tools available for this purpose, for instance Bulk Rename Utility. However, I often find these a bit too complex for what I need.

I have thus developed a little tool – Date Namer – , which does just the one thing I require: To prefix file names with the current date.

date-namer.PNG

Be welcome to download this tool from here:

Upcoming release will be published on this page: Releases. Further, all the source code for this tool is available on GitHub.

For those interested in the implementation details: For this project, I tried using Electron. This allows developing a Desktop application using Node.js. I found this overall quite easy to use. Internally this application will run an instance of Chromium to render the application. The running application takes thus around 50 MB of RAM. I think this is not too bad for this use case. The app performance is very good.

PlantUML (Open Source Awesomeness)

I’ve always had a soft spot for diagrams. I think that representing information in various visual ways tremendously helps our thinking and understanding. Unfortunately it is often a big headache to create (and maintain) diagrams.

So I was very pleased today when I came across PlantUML. PlantUML is a Java library and web service which renders UML diagrams from text input. Take the following text definition for example:


@startuml
object Object01
object Object02
object Object03
object Object04
object Object05
object Object06
object Object07
object Object08

Object01 <|-- Object02
Object03 *-- Object04
Object05 o-- "4" Object06
Object07 .. Object08 : some labels
@enduml

This will be rendered into the following diagram:

diagram

PlantUML does not just support object diagrams but also many other types of diagrams. There is another service, called WebSequenceDiagrams which focusses on only sequence diagrams (and is not open source) but can be useful if more visually pleasing sequence diagrams are required,

Setting Up Graylog Server

I have been looking around for an easy to use and reasonable priced solution for managing logs distributed among many servers and system metrics for these servers. I had a brief look into setting up an ELK system but I found that looked quite cumbersome. Recently I came across Graylog and I found it looked quite promising. I thus set up a little sample system.

While the documentation for Graylog is generally quite good, I found it a bit difficult to piece together the various steps in setting up a minimal working system. Thus I have documented these steps below!

Installing Graylog and Dependencies

Just follow the excellent CentOS installation instructions from the Graylog documetation.

Make sure to provide details for sending emails under the header # Email transport.

If you are using a firewall, open ports 9000 for TCP and 51400 for UPD. For instance, by assuring the following lines are in /etc/sysconfig/iptables.


-A INPUT -p tcp -m state --state NEW -m tcp --dport 9000 -j ACCEPT
-A INPUT -p udp -m state --state NEW -m udp --dport 51400 -j ACCEPT

Don’t forget to restart the iptables service: sudo systemctl restart iptables.

Collecting the Logs from Another CentOS System

  • Install rsyslog on the system

sudo yum install rsyslog

  • Enable and start rsyslog service (also see this guide)

sudo systemctl enable rsyslog

sudo systemctl start rsyslog

  • Edit the file /etc/rsyslog.conf and put the following line at the end, into the section marked as # ### begin forwarding rule ### (replace yourserver.com with your graylog server address.

*.* @yourserver.com:51400;RSYSLOG_SyslogProtocol23Format

  • Restart rsyslog

sudo systemctl restart rsyslog

The rsyslog log messages should now be getting send to your server. Give it a few minutes if you don’t see the messages in graylog immediately. Otherwise, check the system log for any errors (sudo cat /var/log/messages).

Also, you can test the connection by entering the following on the monitored system:


nc -u yourserver.com 51400
Hi

This should result in the message Hi being received by graylog.

Analysing Logs

The next steps are quite easy to to since they can be done in the excellent graylog user interface.

critical errors

  • Create an alert. Trigger it when there is ‘more than 0’ messages in the stream you have just created.

Done! You are now collecting logs from a server and you will receive an email notification whenever there is a serious issue reported on the server!