Configuring an initd Service for node_exporter

I recently wrote an article showing how to configure Prometheus and Grafana for easy metrics collection. In that article, I assumed that the system which should be monitored would use the systemd approach for defining services.

I now had to set up the node_exporter utility on a system which uses the initd approach. Thus, I provide some simple instructions here on how to accomplish that.


  • Extract the archive

tar xvfz node_exporter-*.tar.gz

  • Create a link

ln -s node_exporter-* node_exporter

  • Create the file /opt/node_exporter/ and add the following content:


/opt/node_exporter/node_exporter --no-collector.diskstats

# Provides: node_exporter
# Required-Start: $local_fs $network $named $time $syslog
# Required-Stop: $local_fs $network $named $time $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Description: 



start() {
if [ -f /var/run/$PIDNAME ] && kill -0 $(cat /var/run/$PIDNAME); then
echo 'Service already running' >&2
return 1
echo 'Starting service' >&2
local CMD="$SCRIPT &> \"$LOGFILE\" & echo \$!"
su -c "$CMD" $RUNAS > "$PIDFILE"
echo 'Service started' >&2

stop() {
if [ ! -f "$PIDFILE" ] || ! kill -0 $(cat "$PIDFILE"); then
echo 'Service not running' >&2
return 1
echo 'Stopping service' >&2
kill -15 $(cat "$PIDFILE") && rm -f "$PIDFILE"
echo 'Service stopped' >&2

uninstall() {
echo -n "Are you really sure you want to uninstall this service? That cannot be undone. [yes|No] "
local SURE
read SURE
if [ "$SURE" = "yes" ]; then
rm -f "$PIDFILE"
echo "Notice: log file is not be removed: '$LOGFILE'" >&2
update-rc.d -f  remove
rm -fv "$0"

case "$1" in
echo "Usage: $0 {start|stop|restart|uninstall}"

  • Make both files executable

chmod +x /etc/init.d/node_exporter

chmod +x <em>/opt/node_exporter/</em>

  • Test the script

/etc/init.d/node_exporter start

/etc/init.d/node_exporter stop

  • Enable start with chkconfig

chkconfig --add node_exporter

All done! Now you can configure your Prometheus server to grab the metrics from the node_exporter instance.

Setting up Prometheus and Grafana for CentOS / RHEL 7 Monitoring

As mentioned in my previous post, I have long been looking for a centralised solution for collecting logs and monitoring metrics. I think my search was unsuccessful since I was looking for too many things in one solution. Instead I found now two separate solutions, one for log management (using Graylog) and one for metrics (using Prometheus and Grafana). I deployed both of these on very inexpensive VPS machines and so far I am very happy with them.

In this post, I provide some pointers how to set up the metrics solution based on Prometheus and Grafana. I assume you are using a RHEL / CentOS system as the server hosing Prometheus and Grafana and you are interested in the OS metrics for a CentOS system. This tutorial will guide you through setting up the Prometheus server, collecting metrics for it using node_exporter and finally how to create dashboards and alerts using Grafana for the collected metrics.

Installing Prometheus Server

  • Follow the excellent instructions here with the following modifications.
    • Make sure to download the latest version of Prometheus (the link can be obtained on the Prometheus download page, this guide works with version 2.1.0)
    • For the systemd service, use the following file:

Description=Prometheus Server
ExecStart=/opt/prometheus/prometheus --config.file=/opt/prometheus/prometheus.yml

  • If you are using a firewall, add the following rule to /etc/sysconfig/iptables and restart service iptables:

-A INPUT -p tcp -m state --state NEW -m tcp --dport 9090 -j ACCEPT

Viewing a Metric

You should the data of http requests served by the Prometheus server itself. If you click on the tab graph, you can see the data as a graph.

Installing Node Exporter on the Server to be Monitored

tar xvfz node_exporter-*.tar.gz
  • Create link (replace 0.15.2 with the version you have downloaded)

ln -s node_exporter-0.15.2.linux-amd64 node_exporter

  • Define a service for node_exporter in /etc/systemd/system/node_exporter.service

(or if you are using init.d, please see this article).

Description=Node Exporter

ExecStart=/opt/node_exporter/node_exporter --no-collector.diskstats


(The –no-collector.diskstats is added above since diskstats often does not work in virtualized environments. If that is not an issue for you, be free to leave this argument out.)

  • Enable and start service

systemctl daemon-reload
systemctl start node_exporter
systemctl enable node_exporter

  • Tell Prometheus to scrape these metrics by adding the following to /opt/prometheus/prometheus.yml
 - job_name: "node"
- targets: ['localhost:9100']

Now if you got to you can for instance enter the expression node_memory_MemFree and see the free memory available on the server.

You can also install node_exporter on another server. Simply point the job definition then to this servers address; and of course remember to open port 9100 on the server.

Installing Grafana

The default Prometheus interface is quite basic. Thankfully Grafana offers excellent integration with Prometheus and will result in a much nicer UI.

You can easily install Grafana on your own server or use a free cloud-based instance (limited to one user and five dashboards).

To install Grafana locally:

  • First follow these instructions.
  • Graphana by default runs on port 3000, so make sure you add the following firewall rule after you install it:

-A INPUT -p tcp -m state --state NEW -m tcp --dport 3000 -j ACCEPT

  • In the file /etc/grafana/grafana.ini, provide details for an SMTP connection which can be used for sending emails (section [smtp]).
  • Also update the host name in the field domain to the address at which your server can be reached on the internet.

Configuring Grafana

  • Go to
  • The default login is username ‘admin’ and password ‘admin’. Create a new user with a good password and delete the admin user.
  • First connect with your Prometheus instance as a data source.
  • Then go to Dashboards and select import


Done! You should now be able to see the metrics for your server such as CPU usage or free memory.


If you monitor multiple servers, you can switch between them by clicking next to the text ‘node’.


Additional servers will appear here if you add them to the Prometheus configuration:

- job_name: "node"
 - targets: ['localhost:9100', 'xxxxx']

Configure Alerting

While Prometheus has some build in alerting facilities, alerting in Grafana is much easier to use. To set up altering for the dashboard you have created:

  • Go to Alerting / Notification Channels
  • Click on New Channel
  • Provide a name for the channel and your email address and click Save.


  • Next go to the dashboard you have created: Node Exporter Server Metrics
  • Click on the first panel to select it


  • Next click on Edit in the menu which is shown above the panel
  • Go to the Alert tab and click on Create Alert

create alert

  • Configure the following condition(for more details about this, please see this page):


  • Select the Notification page on the right
  • In Send to, select the notification channel you have created earlier.
  • Provide a message such as: CPU usage high.
  • Save the dashboard (Ctrl+S)

Done! You should now receive notifications if the CPU usage on any of the servers monitored on this dashboard is too high.

Further Reading

Installing Jenkins on Centos 7

I set up a Jenkins server on a brand new Centos 7 VPS. In the following the instructions for doing this in case you are looking at doing the same:

Setting up Jenkins Server

sudo yum install java-1.8.0-openjdk
sudo wget -O /etc/yum.repos.d/jenkins.repo
sudo rpm --import
sudo yum install jenkins

Or for stable version (link did not work for me when I tried it)

sudo wget -O /etc/yum.repos.d/jenkins.repo
sudo rpm --import
yum install jenkins
  • Start Jenkins server
sudo systemctl start jenkins

You should now be able to access Jenkins at (if not, see troubleshooting steps at the bottom).

If you want to access your server more securely on port 80, you can do so by installing ngnix as outlined in this article in step 4: How to Install Jenkins on CentOS 7.

Connecting to a Git Repo

You will probably want to connect to a git repository next. This is also somewhat dependent on the operating system you use, so I provide the steps to do this on CentOS as well:

  • Install git
sudo yum install git
  • Generate an SSH key on the server
ssh-keygen -t rsa
  • When prompted, save the SSH key under the following path (I got this idea from reading the comments here)
  • Assure that the .ssh directory is owned by the Jenkins user:
sudo chown -R jenkins:jenkins /var/lib/jenkins/.ssh
  • Copy the public generated key to your git server (or add it in the GitHub/BitBucket web interface)
  • Assure your git server is listed in the known_hosts file. In my case, since I am using BitBucket my /var/lib/jenkins/.ssh/known_hosts file contains something like the following, ssh-rsa [...]
  • You can now create a new project and use Git as the SCM. You don’t need to provide any git credentials. Jenkins pulls these automatically form the /var/lib/jenkins/.ssh directory. There are good instructions for this available here.

Connecting to GitHub

  • In the Jenkins web interface, click on Credentials and then select the Jenkins Global credentials. Add a credential for GitHub which includes your GitHub username and password.
  • In the Jenkins web interface, click on Manage Jenkins and then on Configure System. Then scroll down to GitHub and then under GitHub servers click the Advanced Button. Then click the button Manage additional GitHub actions.

additional actions

  • In the popup select Convert login and password to token and follow the prompts. This will result in a new credential having been created. Save and reload the page.
  • Now go back to the GitHub servers section and now click to add an additional server. As credential, select the credential which you have just selected.
  • In the Jenkins web interface, click on New Item and then select GitHub organisation and connect it to your user account.

Any of your GitHub projects will be automatically added to Jenkins, if they contain a Jenkinsfile. Here is an example.

Connect with BitBucket

  • First, you will need to install the BitBucket plugin.
  • After it is installed, create a normal git project.
  • Go to the Configuration for this project and select the following option:

BitBucket trigger

  • Log into BitBucket and create a webhook in the settings for your project pointing to your Jenkins server as follows: (note the slash at the end)

Testing a Java Project

Chances are high you would like to run tests against a Java project, in the following, some instructions to get that working:


  • If you cannot open the Jenkins web ui, you most likely have a problem with your firewall. Temporarily disable your firewall with: `sudo systemctl stop iptables` and see if it works then.
  • If it does, you might want to check your rules in `/etc/sysconfig/iptables` and assure that port 8080 is open
  • Check the log file at:
sudo cat /var/log/jenkins/jenkins.log


Upgrade MySQL 5.5 to 5.6 on EC2/CentOS/RHEL


You would like to upgrade MySQL 5.5 to MySQL 5.6 on an existing server that uses the YUM software package manager.


Just enter the following few simple commands and you should be good to go. But, please, do a thorough full backup of your system before you do the upgrade just in case.

[1] Create a MySQL dump from which you will load the data into the upgraded server:

mysqldump -u root -p –add-drop-table –routines –events –all-databases –force > data-for-upgrade.sql

[2] Stop your MySQL server

sudo service mysqld stop

[3] Remove MySQL 5.5

yum remove mysql55-server mysql55-libs mysql55-devel mysql55-bench mysql55

[4] Clear the MySQL data directory

sudo rm -r /var/lib/mysql/*

[5] Install MySQL 5.6

sudo yum install mysql56 mysql56-devel mysql56-server mysql56-libs

[6] Start MySQL server

sudo service mysqld start

[7] Set the root password

/usr/libexec/mysql56/mysqladmin -u root password ‘xxx’

[8] Import your data

mysql -u root -p –force < data-for-upgrade.sql

[9] Verify all tables will work in 5.6

sudo mysql_upgrade -u root -p –force

All done!


  • Upgrade to 5.7 should work in a similar way, once 5.7 is available on your RPM repos (it isn’t by the time of the writing for the Amazon Linux Repo).






Cannot Install VirtualBox Guest Additions: Installer Hangs


For a CentOS based guest, the update of the VirtualBox Guest Additions hangs after the step: “Removing exiting VirtualBox non-DKMS kernel modules”



  • Check the log file in /var/log/VBoxGuestAdditions-uninstall.log. See if there is anything obvious reported there you can fix.
  • Check if your RAM disks are valid by running the command `sudo lsinitrd`.
    • If they are not, try rebuilding broken images with `sudo dracut -f [broken-image].img [kernel version]`
  • Verify you have all dependencies installed:
    • yum install dkms
    • yum groupinstall “Development Tools”
    • yum install kernel-devel
  • Reinstall your Kernel image
  • Verify that you have the correct version of kernel-devel installed
    • yum remove kernel-devel
    • yum install kernel-devel-`uname -r`
    • Restart system!
  • Wait!
    • Let the installation run for a while, at least 15-20 min. It might do something!

newrelic service doesn’t start on CentOS: Solution Run as Root

Today one of my virtual servers suddenly showed up as offline in my New Relic console (Which is an amazing tool and free by the way).

I checked the log file /var/log/newrelic/nrsysmond.log and it had these contents:

2015-08-15 02:06:04.512 (915) error: nria_context_create(): SIGAR_OK != status; goto error; status=13

2015-08-15 02:06:04.512 (915) error: nria_context_create(): in error label

2015-08-15 02:06:04.512 (915) error: failed to create sampling context

2015-08-15 02:06:04.512 (913) info: worker process exited with 1 – NOT restarting

I found a post on the New Relic forum which provided the solution:

-> Run the newrelic service as root

This can be done by editing /etc/sysconfig/newrelic-sysmond and uncommenting the option RUNAS:

# User to run the Server Monitor as

# RUNAS=newrelic

Not a perfect solution since ideally this should run with the newrelic user but at least it got the server online on my console again.

Access Apache on VirtualBox OS from Host OS


You have a RHEL or CentOS 6 VirtualBox Virtual Machine and a Windows or other host and would like to access a web server on the VM from the Host OS.


Assure that the Apache Web Server is available and started on the guest OS:

`sudo httpd start`

Assure that firewall allows the guest OS to handle web requests.

Type ` sudo system-config-firewall` and enable the WWW service.

Configure port forwarding for the VM. Go to the VM settings in the Oracle VM VirtualBox Manager / Page ‘Network’.

Click on the button [Port Forwarding] and add an entry such as the following:

Now, on your host OS, you should be able to see the Apache Server test page when opening `http://localhost:8080` in a browser.


Since every system is configured differently there might be some problems you encounter. If you don’t see the test page, you could try the following:

  • Disable SELinux by opening the file ` /etc/selinux/config` and setting the configuration to ` SELINUX=disabled` (see a more detailed guide here)
  • This tutorial assumes that you use the default NAT network adapter for you VM. If you use a ‘host-only’ or ‘bridged’ connection, the process will be a bit different.


Stackoverflow – How to SSH to a VirtualBox guest externally through a host? Very helpful, also read the comments

Stackoverflow – Apache restart causes DocumentRoot must be a directory, even though it is a directory and there seem to be no privilege issues

CentOS Board – Basic firewall config (just port 80 and ssh 22)