Advantages of Using React Hooks

I always had the feeling that React is just a bit to complex, a bit to ‘heavy’ to be a truely elegant solution to the problem of building complex user interfaces in JavaScript. Two issues, for instance, are the general project setup, exemplified by the need to have create-react-app, and class-based components, with all their componentDidMount and this references.

While React Hooks are no solution to the first issue, they provide, in my mind, an elegant solution to the second; they provide a better way to do what we used to do with class-based components.

To illustrate this, I will first provide an implementation of a simple component using a class-based component and then refactor this into an implementation using React Hooks.

Here the initial implementation using a class-based component:

class User1 extends Component {
  constructor(props) {
    super(props);

    this.state = {
      userId: props.userId,
      userName: null,
      isLoading: false,
      error: null,
      unmounted: false,
    };
  }

  getUser() {
    this.setState({ isLoading: true, error: null });
  
    axios.get(`https://jsonplaceholder.typicode.com/users/${this.state.userId}`)
      .then(result => {
        if (this.state.unmounted) {
          return;
        }
        this.setState({
          userName: result.data.name,
          isLoading: false
        })
      }
      )
      .catch(error => {
        if (this.state.unmounted) {
          return;
        }

        this.setState({
          error,
          isLoading: false
        })
      });
  }

  componentDidMount() {
    this.getUser();
  }

  componentDidUpdate() {
    // this.getUser();
  }

  componentWillUnmount() {
    this.setState({ unmounted: true });
  }

  render() {
    return (<>
      {this.state.isLoading ? <p>Loading ...</p> : <></>}
      {this.state.error ? <p>Cannot load user</p> : <></>}
      {!this.state.isLoading && !this.state.error ? <p>{this.state.userName}</p> : <></>}
      <button onClick={() => {
        const newUserId = this.state.userId + 1;
        this.setState({ userId: newUserId }, this.getUser);
        this.getUser();
      }} >Next</button>
    </>);
  }
}

As can be seen in above code, this component requests data about a user from JSONPlaceholder and then display this data. There is also a button that will trigger loading of another user.

Simple enough – but we still need a fair amount of code to handle this scenario in a robust manner, including instances where we start the request for a new user before the previous request has been completed or where a request only completes after the component has been unmounted.

A component with the exact same functionality can be implemented using React Hooks:

function User2(props) {
  const [userId, setUserId] = useState(props.userId);
  const [name, setName] = useState(null);
  const [isLoading, setIsLoading] = useState(true);
  const [isError, setIsError] = useState(false);

  useEffect(() => {
    let cancelled = false;
    const fetchData = async () => {
      setIsLoading(true);
      setIsError(false);
      let response;
      try { 
        response = await axios.get(`https://jsonplaceholder.typicode.com/users/${userId}`);
      } catch (e) {
        setIsError(true);
        setIsLoading(false);
        return;
      }
      setIsLoading(false);
      if (cancelled) return;
      setName(response.data.name);
    };
    fetchData();
    return () => {
      cancelled = true;
    };
  }, [userId]);

  return (<>
    {isLoading ? <p>Loading ...</p> : <></>}
    {isError ? <p>Cannot load user</p> : <></>}
    {!isLoading && !isError && name ? <p>{name}</p> : <></>}
    <button onClick={() => setUserId(userId + 1)} >Next</button>
  </>);
}

Here we use useState to define a number of state variables and useEffect to deal with state updates. useState of course is essential in allowing us to define a functional component that also uses state. One major advantage in my mind of the hooks-based approach is that we don’t need to worry about using this and are in no danger of forgetting it.

useEffect replaces the functionality of componentDidMount and componentDidUpdate in the class-based components. I think it allows reacting to state changes in a much more elegant way. Firstly by linking it to the state of userId the useEffect handler we have defined will only trigger when the userId status has been updated, without us having to add any additional tests and logic around that. Secondly, it elegantly handles both the cases for when the component mounts as well as when the component state changes: by always triggering on component mount, and subsequently on changes to the userId. Thirdly, by returning a function as the result of the useEffect handler …

    return () => {
      cancelled = true;
    };

… we have a very easy way to deal with the component unmounting when a request is in flight.

However, the real power of React Hooks, in my mind, lies in the composability of Hooks. The following example implements the features for the component using a custom open source hook: use-data-api:

import useDataApi from 'use-data-api';

function User3(props) {
  const [userId, setUserId] = useState(props.userId);
  const [{ data, isLoading, isError }, performFetch] = useDataApi(null, null);

  useEffect(() => {
    performFetch(`https://jsonplaceholder.typicode.com/users/${userId}`);
  }, [userId, performFetch]);

  return (<>
    {isLoading ? <p>Loading ...</p> : <></>}
    {isError ? <p>Cannot load user</p> : <></>}
    {!isLoading && !isError && data ? <p>{data.name}</p> : <></>}
    <button onClick={() => setUserId(userId + 1)} >Next</button>
  </>);
}

Above we use the custom hook useDataApi that takes care of the details of having to deal with requests to an API (use-data-api/blob/master/src/index.js).

As can be seen, this last example is much shorter in length and easier to understand than the previous examples. Thus showing the biggest advantage for React Hooks – to extract complex behaviour into external functions that can be easily reused within a project and across projects.

To summarise, here all the advantages of using React Hooks discussed above:

  • Ability to create composite Hooks defining cross-cutting functionality concerns in an application.
  • Enables writing functional components with state (no more this).
  • useEffect provides a more concise and elegant way to handle component mount, update and unmount events.

Here the complete source code code of the examples used in this post:

react-hooks-tutorial

GraphQL, Node.JS and React Monorepo Starter Kit

Following the GraphQL Apollo Starter Kit (Lerna, Node.js), I wanted to dig deeper into developing a monorepo for a GraphQL/React client-server application.

Unfortunately, things are not as easy as I thought at first. Chiefly the create-react-app template does not appear to work very well with monorepos and local dependencies to other packages.

That’s why I put together a small, simple starter template for developing modular client-server applications using React, GraphQL and Node.js. Here is the code on GitHub:

nodejs-react-monorepo-starter-kit

Some things to note:

  • There are four packages in the project
    • client-main: The React client, based on create-react-app
    • client-components: Contains a definition of the component app. Used by client-main
    • server-main: The Node.js server definition
    • server-books: Contains schema and resolver for GraphQL backend. Used by server-main.
  • Each package defines it’s own package.json and can be built independent of the other packages.
  • The main entry point for the dependent packages (client-components and server-books) is set to dist/index.js. This way, packages which use them, can use the transpiled version created by babel and don’t need to worry about specific JS features used in the dependent packages.

Like GraphQL Apollo Starter Kit (Lerna, Node.js) this starter kit is meant to be very basic to allow easy exploration of the source code.

Mastering Modular JavaScript

Today I was having a look around for best practices for defining JavaScript modules. In that search, I came across the book Mastering Modular JavaScript. This book offers a good selection of best practices for JS module development. Also, all chapters are freely available on GitHub:

For a more basic introduction to modules, see the chapter JavaScript Modules from the book Practical Modern JavaScript.

Everything new in JavaScript since ES6

It is no secret that things in the tech world change rather rapidly. It’s difficult to keep track of everything at the same time. For instance I have been working with JavaScript quite extensively some years ago but recently have been more involved with other tech stacks. Thus I have only followed the developments in the JavaScript world sporadically and was quite surprised by how many things have changed since the days of JavaScript: The Good Parts.

Since before ES6 things have not changed much for a long time, I imagine I am not the only one who could benefit from a little refresher of all the things that have changed since ES6. Thus I have compiled some of the changes I think are most important for ordinary development work. The idea is to provide a quick overview rather than explain every feature in detail – assuming that more information on any of the changes is readily available on the web.

This is not a complete list of everything that has changed. For instance, I included promises but omitted changes made to the way regular expressions work in ECMAScript 2018; since we are likely to come across promises many times per day whereas the changes to regular expressions only affect us in particular edge cases.

ECMAScript 6 / ECMAScript 2015

Variable Scoping

  • let x = 1;: To define block scoped variables

Arrow Functions

  • x => x + 1: Concise closure syntax
  • x => { return x + 1; }: Concise closure syntax
  • this: within lambdas refers to enclosing object (rather than to lambda function itself)

Promises

Promises for wrapping asynchronous code.


let p = new Promise((resolve, reject) => {

   resolve("hello");

});

p.then((msg) => console.log(msg)); 

Executing asynchronous operations in parallel

let parallelOperation = Promise.all([p1, p2]);
parallelOperation.then((data) => {let [res1, res2] = data; } );

Default Parameters and Spread Operator

  • function (x = 1, y = 2): Default values for function parameters
  • function (x, y, ...arr) {}: Capturing all remaining arguments in array for variadic functions
  • var newarr = [ 1, 2, ...oldarr]: ‘Spreading’ of elements from an array as literal elements
  • multiply(1, 2, ...arr): Spreading of elements from an array as individual function parameters

Multiline Strings and Templates

  • `My String⏎NewLine`: Multi-line string literals
  • `Hello ${person.name}`: Intuitive string interpolation
  • const proc = sh`kill -9 ${pid}`;: Tagged template literals for parsing custom languages. The example would result in calling the function sh with the parameters (['kill -9 '], pid)

Object Properties

  • let obj = { x, y }: Property shorthand for defining let object = { x: x, y: y }
  • obj = { func1 (x, y) { } }: Methods allowed as object properties

Deconstructor Assignment

  • var [ x, y, z ] = list: Deconstructing arrays into individual variables by assignment.
  • var [ x=0, y=0 ]: Default values for deconstructing arrays.
  • function( [ x, y ] ): Deconstructing arrays in function calls.
  • var { x, y, z } = getPoint(): Deconstructing objects into individual variables by assignment.
  • var { name: name, address: { street: street }, age: age} = getData(): Deconstructing objects into individual variables by assignment, including nested properties.
  • var p = { x=0, y=0 }: Default values for deconstructing objects.
  • function( { x, y } ): Deconstructing objects in function calls.

Modularity

  • export function add(x,y) { return x + y; }: Exporting functions
  • export var universe = 42;: Exporting variables
  • import { add, universe } from 'lib/module';: Importing functions and variables
  • import * from 'lib/module': Wildcard import
  • export default (x, y) => x + y;: Defining default export
  • import add from 'lib/add': Importing default export
  • import add, { universe } from 'lib/add': Importing default export and additional exports
  • export * from 'lib/module';: Reexporting from other modules

Classes

class keyword for constructing simple classes.

class Point {

  constructor (x, y) {
     this.x = x;
     this.y = y;
  }

  move (deltax, deltay) {
     new Point(this.x + deltax, this.y + deltay);
  }

}

extends keyword for extending classes:


class Car extends Vehicle {

  constructor (name) {
     super(name);
  }

static keyword for static methods


class Math {

  static add(x, y) {
    return x + y;
  }

}

get and set keywords for decorated property access.


class Rectangle {

  get area() { return this.x * this.y }

}

...

new Rectangle(2, 2).area === 4;

Iteration Through Object Values

  • for (let value of arr) { }: for … of loop for going iterating through values of objects.
  • Also note that objects can define their own iterators and generators

Data Structures

  • new Set(): For sets
  • new Map(): For maps
  • new WeakSet(): For sets whose items will be garbage collected when required
  • new WeakMap(): For sets whose items will be garbage collected when required

Symbols

  • Symbol(): For creating an object with a unique identity.
  • Symbol("note"): For creating a unique object with a descriptor.
  • Note: Symbol("node") !== Symbol("node")

ECMAScript 2016

  • **: Exponentiation operator
  • Array.prototypes.includes: Like indexOf but with true/false result and support for NaN

ECMAScript 2017

async/await for more expressive asynchronous operations

async function add1(x) {
  return x + 1;
}

async function add2(x) {
  let y = await add1(x);
  return await add1(y);
}

add2(5).then(console.log);

ECMAScript 2018

Rest/Spread Operators for Object Properties

Collect all not deconstructed properties from an object in another object:


var person = { firstName: "Paul", lastName: "Hendricks", password: "secret"};
var {password, ...sanitisedPerson } = person;
// sanitisedPerson = {firstName: "Paul", lastName: "Hendricks"}

Spread object properties

let details = { firstName: "Paul", lastName: "Hendricks" };

let user = { ...details, password: "secret" };

Finally for Promises

finally callback is guaranteed to be executed if promise succeeds or fails.


async function sayHello() {
console.log("hello");
}
sayHello().then(() => console.log("success") )
.catch((e) => console.log(e))
.finally(() => console.log("runs always")

for await Loop

Special for loops that resolve promises before every iteration.


const promises = [
  new Promise(resolve => resolve(1) ),
  new Promise(resolve => resolve(2) )
];

async function runAll() {
  for await (p of promises) {
    console.log(p);
  }
}

runAll();

References

Image credits: Flickr

Designing Micro Services the Right Way

For a few years now, micro services have been all the rage when it comes to the architecture of large applications. Personally I have always been a bit puzzled about what was so new a great about micro services in comparison to what came before them: Service Oriented Architecture (SOA). Indeed, SOA itself is often portrayed as a frightful antipattern from our past to be mentioned in the same breath as CORBA.

To me, the move from CORBA et all to SOA to Micro Services has not been one of disrupting innovation but one of continuous learning; chiefly in relation to the technologies we employ. It just makes a world of difference setting up a big old monolithic application from the past or an express server in Node.JS (which is also a ‘monolith’ in its own right but just a smaller one – hopefully).

The core problem we are trying to solve has not changed: distributed computing. Unfortunately, one of the first things we learnt about distributed computing seems to have been given less attention recently: that it is best avoided wherever possible. Why? Because it introduces great complexity into an application and can result in many development and operational problems (see YouTube: 10 Tips for failing badly at Microservices by David Schmitz).

One of the most problematic areas is data or persisted state. If the same piece of data needs to be used by multiple services, things become very complicated since it is often required to keep data in sync between multiple places (see YouTube: Managing Data in Microservices by Randy Shoup).

Recently I came across a presentation which I think outlined a very nice approach for dealing with micro services – one that relied heavily on code generation, enforcing common standards and automated testing. Furthermore in the presented architecture one language was used primarily, which I think is a very good approach. I highly recommend viewing this presentation for anyone interested in a way to deal with the complexity of micro services:

YouTube: Design Microservice Architectures the Right Way by Michael Bryzek

What I personally took away from this:

  • Focus on testability. Allow for fast unit and integration tests and even testing with production data. Only code that is easy to test and heavily tested allows for fast and bold development. This organisation for instance automatically updates all their dependencies once per week – automatically, since they have full confidence that their tests will pick up any issues.
  • Utilise code generation. The sad truth of micro services is that we will have to duplicate things, such as commonly used entities – especially if multiple programming languages are involved. Code generation provides an elegant way to deal with this unfortunate situation.
  • Enforce common standards. Although micro services are intended to reduce complexity by dividing up a complex system into small manageable chunks, they can actually result in increased overall complexity, especially if many different technologies are employed. In that case, enforcing strict common standards can help in keeping things simple for developers and ops.
  • Embrace events. Triggering services into action by using events rather than direct API calls can help in making a distributed system more predictable and easier to debug.

I think this presentation provides an excellent overview of best practices for micro services and I couldn’t think of anything to criticise or add. I think it represents the best way of building micro services I am aware of as of now.

I do think, however, there is one important additional issue to consider, and that is that a micro service built according to the best principles and standards will still be a liability if it wasn’t necessary to build a micro service to begin with. This is not so much the question if we should micro services or not (in any organisation of a certain size they are an imperative) but how many.

One of the key drivers of success for micro services within a larger system is to get the boundaries of the services right (see bounded context) and I think we should aim to make micro services as large as possible so we have as few of them as possible; taking into consideration the restrictions of team size, data and complexity:

  • Team: It might sound like heresy but I do think that one ‘physical’ micro service could be maintained by up to three to five teams (and not just one team per micro service). That of course would be the upper maximum, there is nothing wrong with having just one team per micro service. It really depends on what service you are building.
  • Data: And some more heresy: I think that for data it is often better to scale up rather than scale out. Why? Data is all about state and being able to keep state within the physical confines of one systems leads to much improved performance and reduced complexity. Thus we should think about the database management system we will be using for our service and what is the maximum we can scale it up to. Then take 20% of that and ask yourself if your data will stay within that limit. If not, it might be prudent to break the micro service apart or maybe change the DBMS.
  • Complexity: The main drivers of complexity in software are code size, inter-dependencies and heterogeneity. If our micro service would contain large amounts of code with many intricate inter-dependencies that tackles many different problems in different ways, it may be advisable to think about breaking the service up.

As mentioned, distributed systems are inherently more complex that non-distributed ones. Therefore, if we have larger micro services, our system becomes less distributed overall and we hopefully have less accidental complexity to deal with.

Thus, to sum things up, we must be aware of the dangers of micro services and deploy tooling strategically as outlined in the presentation as well as be mindful of how we can build our system in a way that we avoid the complexities of distributed systems as much as possible.

Setting up Prometheus and Grafana for CentOS / RHEL 7 Monitoring

As mentioned in my previous post, I have long been looking for a centralised solution for collecting logs and monitoring metrics. I think my search was unsuccessful since I was looking for too many things in one solution. Instead I found now two separate solutions, one for log management (using Graylog) and one for metrics (using Prometheus and Grafana). I deployed both of these on very inexpensive VPS machines and so far I am very happy with them.

In this post, I provide some pointers how to set up the metrics solution based on Prometheus and Grafana. I assume you are using a RHEL / CentOS system as the server hosing Prometheus and Grafana and you are interested in the OS metrics for a CentOS system. This tutorial will guide you through setting up the Prometheus server, collecting metrics for it using node_exporter and finally how to create dashboards and alerts using Grafana for the collected metrics.

Installing Prometheus Server

  • Follow the excellent instructions here with the following modifications.
    • Make sure to download the latest version of Prometheus (the link can be obtained on the Prometheus download page, this guide works with version 2.1.0)
    • For the systemd service, use the following file:

[Unit]
Description=Prometheus Server
Documentation=https://prometheus.io/docs/introduction/overview/
After=network-online.target
[Service]
User=root
Restart=on-failure
ExecStart=/opt/prometheus/prometheus --config.file=/opt/prometheus/prometheus.yml
[Install]
WantedBy=multi-user.target

Note: This configuration will run the Prometheus server as root. In a production environment, it is highly recommended to run it as another user (e.g. ‘prometheus’)

  • If you are using a firewall, add the following rule to /etc/sysconfig/iptables and restart service iptables:

-A INPUT -p tcp -m state --state NEW -m tcp --dport 9090 -j ACCEPT

Viewing a Metric

You should the data of http requests served by the Prometheus server itself. If you click on the tab graph, you can see the data as a graph.

Installing Node Exporter on the Server to be Monitored

tar xvfz node_exporter-*.tar.gz
  • Create link (replace 0.15.2 with the version you have downloaded)

ln -s node_exporter-0.15.2.linux-amd64 node_exporter

  • Define a service for node_exporter in /etc/systemd/system/node_exporter.service

(or if you are using init.d, please see this article).


[Unit]
Description=Node Exporter

[Service]
User=root
ExecStart=/opt/node_exporter/node_exporter --no-collector.diskstats

[Install]
WantedBy=default.target

Note: This configuration will run the Grafana server as root. In a production environment, it is highly recommended to run it as another user (e.g. ‘prometheus’)

(The –no-collector.diskstats is added above since diskstats often does not work in virtualized environments. If that is not an issue for you, be free to leave this argument out.)

  • Enable and start service

systemctl daemon-reload
systemctl start node_exporter
systemctl enable node_exporter

  • Tell Prometheus to scrape these metrics by adding the following to /opt/prometheus/prometheus.yml
 - job_name: "node"
static_configs:
- targets: ['localhost:9100']

Now if you got to yourserver.com:9090/graph you can for instance enter the expression node_memory_MemFree and see the free memory available on the server.

You can also install node_exporter on another server. Simply point the job definition then to this servers address; and of course remember to open port 9100 on the server.

Installing Grafana

The default Prometheus interface is quite basic. Thankfully Grafana offers excellent integration with Prometheus and will result in a much nicer UI.

You can easily install Grafana on your own server or use a free cloud-based instance (limited to one user and five dashboards).

To install Grafana locally:

  • First follow these instructions.
  • Graphana by default runs on port 3000, so make sure you add the following firewall rule after you install it:

-A INPUT -p tcp -m state --state NEW -m tcp --dport 3000 -j ACCEPT

  • In the file /etc/grafana/grafana.ini, provide details for an SMTP connection which can be used for sending emails (section [smtp]).
  • Also update the host name in the field domain to the address at which your server can be reached on the internet.

Configuring Grafana

  • Go to yourserver.com:3000
  • The default login is username ‘admin’ and password ‘admin’. Create a new user with a good password and delete the admin user.
  • First connect with your Prometheus instance as a data source.
  • Then go to Dashboards and select import

dashboards

Done! You should now be able to see the metrics for your server such as CPU usage or free memory.

dashboard

If you monitor multiple servers, you can switch between them by clicking next to the text ‘node’.

switch

Additional servers will appear here if you add them to the Prometheus configuration:

- job_name: "node"
 static_configs:
 - targets: ['localhost:9100', 'xxxxx']

Configure Alerting

While Prometheus has some build in alerting facilities, alerting in Grafana is much easier to use. To set up altering for the dashboard you have created:

  • Go to Alerting / Notification Channels
  • Click on New Channel
  • Provide a name for the channel and your email address and click Save.

channel

  • Next go to the dashboard you have created: Node Exporter Server Metrics
  • Click on the first panel to select it

panel

  • Next click on Edit in the menu which is shown above the panel
  • Go to the Alert tab and click on Create Alert

create alert

  • Configure the following condition(for more details about this, please see this page):

condition

  • Select the Notification page on the right
  • In Send to, select the notification channel you have created earlier.
  • Provide a message such as: CPU usage high.
  • Save the dashboard (Ctrl+S)

Done! You should now receive notifications if the CPU usage on any of the servers monitored on this dashboard is too high.

Further Reading

Determine Which JDK Version a JAR/Class File Was Compiled With

Today I came across a nasty error which occurred in a deployed Java application only but not during development or integration tests. The error went something like the following:

java.lang.NoSuchMethodError: java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;
at nx.serializerkryo.internal.InternalKryoSerialzer.performToStream(InternalKryoSerialzer.java:33)
at nx.serializerkryo.internal.InternalKryoSerialzer.serialize(InternalKryoSerialzer.java:63)
at nx.serializerkryo.internal.InternalKryoSerialzer.serialize(InternalKryoSerialzer.java:21)
at nx.persistence.jre.internal.OptimizedPersistedNodeSerializer.serialize(OptimizedPersistedNodeSerializer.java:47)
at nx.persistence.jre.internal.OptimizedPe<span 				data-mce-type="bookmark" 				id="mce_SELREST_end" 				data-mce-style="overflow:hidden;line-height:0" 				style="overflow:hidden;line-height:0" 			></span>rsistedNodeSerializer.serialize(OptimizedPersistedNodeSerializer.java:21)

Now I had a feeling that this had something to do with me trying to be ahead of the curve and use a Java 9 JDK to compile the application. In order to debug this, I had to confirm which with JDK the classes I was using were compiled with. Thankfully I found a handy thread of StackOverflow.

Unfortunately, it wasn’t immediately obvious to me which solution listed there would work best, so I decided to provide the solution here in a more condensed form. Simply use the following command:

javap -v [path to your class file]

The output will then contain the following line (towards the top of the file):

public class ...
minor version: 0
major version: 50
flags: ACC_PUBLIC, ACC_SUPER

The major version and minor version indicates which version of Java the class was compiled with. The following contains a list of which Java versions which major versions relate to.

Java SE 9 = 53 (0x35 hex),
Java SE 8 = 52 (0x34 hex),
Java SE 7 = 51 (0x33 hex),
Java SE 6.0 = 50 (0x32 hex),
Java SE 5.0 = 49 (0x31 hex),
JDK 1.4 = 48 (0x30 hex),
JDK 1.3 = 47 (0x2F hex),
JDK 1.2 = 46 (0x2E hex),
JDK 1.1 = 45 (0x2D hex).

Interestingly my files were apparently compiled for Java 6 (Maven compiler plugin was responsible). The problem was that the files were compiled with JDK 9 (though they were compiled for 1.6). Downgrading the JDK used to do the compilation to JDK8 fixed the problem.

Installing Jenkins on Centos 7

I set up a Jenkins server on a brand new Centos 7 VPS. In the following the instructions for doing this in case you are looking at doing the same:

Setting up Jenkins Server

sudo yum install java-1.8.0-openjdk
sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo
sudo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key
sudo yum install jenkins

Or for stable version (link did not work for me when I tried it)

sudo wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo
sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
yum install jenkins
  • Start Jenkins server
sudo systemctl start jenkins

You should now be able to access Jenkins at yourserver.com:8080 (if not, see troubleshooting steps at the bottom).

If you want to access your server more securely on port 80, you can do so by installing ngnix as outlined in this article in step 4: How to Install Jenkins on CentOS 7.

Connecting to a Git Repo

You will probably want to connect to a git repository next. This is also somewhat dependent on the operating system you use, so I provide the steps to do this on CentOS as well:

  • Install git
sudo yum install git
  • Generate an SSH key on the server
ssh-keygen -t rsa
  • When prompted, save the SSH key under the following path (I got this idea from reading the comments here)
/var/lib/jenkins/.ssh
  • Assure that the .ssh directory is owned by the Jenkins user:
sudo chown -R jenkins:jenkins /var/lib/jenkins/.ssh
  • Copy the public generated key to your git server (or add it in the GitHub/BitBucket web interface)
  • Assure your git server is listed in the known_hosts file. In my case, since I am using BitBucket my /var/lib/jenkins/.ssh/known_hosts file contains something like the following
bitbucket.org,104.192.143.3 ssh-rsa [...]
  • You can now create a new project and use Git as the SCM. You don’t need to provide any git credentials. Jenkins pulls these automatically form the /var/lib/jenkins/.ssh directory. There are good instructions for this available here.

Connecting to GitHub

  • In the Jenkins web interface, click on Credentials and then select the Jenkins Global credentials. Add a credential for GitHub which includes your GitHub username and password.
  • In the Jenkins web interface, click on Manage Jenkins and then on Configure System. Then scroll down to GitHub and then under GitHub servers click the Advanced Button. Then click the button Manage additional GitHub actions.

additional actions

  • In the popup select Convert login and password to token and follow the prompts. This will result in a new credential having been created. Save and reload the page.
  • Now go back to the GitHub servers section and now click to add an additional server. As credential, select the credential which you have just selected.
  • In the Jenkins web interface, click on New Item and then select GitHub organisation and connect it to your user account.

Any of your GitHub projects will be automatically added to Jenkins, if they contain a Jenkinsfile. Here is an example.

Connect with BitBucket

  • First, you will need to install the BitBucket plugin.
  • After it is installed, create a normal git project.
  • Go to the Configuration for this project and select the following option:

BitBucket trigger

  • Log into BitBucket and create a webhook in the settings for your project pointing to your Jenkins server as follows: http://youserver.com/bitbucket-hook/ (note the slash at the end)

Testing a Java Project

Chances are high you would like to run tests against a Java project, in the following, some instructions to get that working:

Troubleshooting

  • If you cannot open the Jenkins web ui, you most likely have a problem with your firewall. Temporarily disable your firewall with: `sudo systemctl stop iptables` and see if it works then.
  • If it does, you might want to check your rules in `/etc/sysconfig/iptables` and assure that port 8080 is open
  • Check the log file at:
sudo cat /var/log/jenkins/jenkins.log

 

Versioning WordPress with Git and Revisr

WordPress is a powerful platform to just get a simple website up and running. Unfortunately, some things which are considered best practice in software development projects are a bit difficult to realize with WordPress. One of these things is versioning. Thankfully, there is a powerful plug in which enables versioning WordPress using a git repository: Revisr.

As of writing this, one is first greeted by an error message when visiting the Revisr website (something to do with SSL). It is safe to ignore this and to instruct your browser to show this website irrespective of the error message displayed (you won’t be giving any confidential information to this webiste, just browsing around).

In any case, you can download Revisr from WordPress with the following link:

https://wordpress.org/plugins/revisr/

Following the steps for setting it up:

  • Go to your WordPress Admin console
  • Install the plugin
  • Activate it
  • Reload  WordPress Admin console
  • Click on Revisr on the Sidebar
  • Instruct it to create a new repository

That’s already it for setting up a local git repo that will enable some versioning. However, you can also use this plugin to backup your site and all versions to a remote git repository, for instance using BitBucket. The instructions for this come as follows (assuming you are using an Apache Web server):

  • Login to your server using SSH with a user with sudo rights
  • Execute the following
sudo ssh keygen
  • Follow the prompts to create the key
  • Execute the following (where /var/www is the root dir of your Apache server)
sudo cp -r /root/.ssh /var/www

sudo chown -R apache:apache /var/www/.ssh
  • Create the file /var/www/.ssh/.htaccess and put in the following content (this is just a security measure)
Deny from all
  • Grab the public key and save it somewhere
sudo cat /var/www/.ssh/id_rsa.pub
  • Create a new account for BitBucket if you don’t have one already.
  • Add a public SSH key for your account. Add the SSH key you saved earlier.
  • Create a new repository. Grab the git SSH link for this repository.
  • Go back to your WordPress Admin console and select the Revisr plugin from the sidebar
  • Go to Settings / General. Set your username to git and define an email. Click Save
  • Go to Settings / Remote. Set Remote URL to the SSH link for the repository you saved earlier. Click Save.

Now you can go back to the main Revisr page and start committing changes and pushing to the remote repository!

 

Resize EC2 Volume (without Resizing Partition)

Problem

You would like to resize a volume attached to an EC2 instance.

Solution

Do the following:

  • Create a snapshot of your volume (instructions)
  • Stop your instance
  • Go to EBS / Volumes and select Actions / Modify Volume

Modify Vol

  • Enter the new size for your volume (note you can only ever make the volume larger) and click on Modify

size

  • Wait for the modification to be complete (this might take a while, like 30 min or so)
  • Start your instance

Now, if everything went well, you should have more space available on the disk for the virtual machine. To confirm this, run:

df -h

You should see the new size of the volume as the size of your main partition:

size2

Notes

  • If the size of your partition, does not match the size of the volume, you probably need to resize your partition (instructions).
  • Resizing the partition is a very painful process, that I think should best be avoided at all costs. I think for this it helps if the EC2 instance attached to the volume is stopped when the resize is performed. Assure that this is the case before you do the resize.
  • If you forgot to stop your instance, and need to do a partition resize, there is a little workaround. Wait for six hours, then resize your volume again (this time while the instance is stopped). Then, it hopefully adjusts your partition size to the correct size.
  • In the above, you might be able to start up your instance even while the new volume is still optimizing. I haven’t tested this though but my guess is that it would work.