Tag: Jenkins

Installing Jenkins on Centos 7

I set up a Jenkins server on a brand new Centos 7 VPS. In the following the instructions for doing this in case you are looking at doing the same:

Setting up Jenkins Server

sudo yum install java-1.8.0-openjdk
sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo
sudo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key
sudo yum install jenkins

Or for stable version (link did not work for me when I tried it)

sudo wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo
sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
yum install jenkins
  • Start Jenkins server
sudo systemctl start jenkins

You should now be able to access Jenkins at yourserver.com:8080 (if not, see troubleshooting steps at the bottom).

If you want to access your server more securely on port 80, you can do so by installing ngnix as outlined in this article in step 4: How to Install Jenkins on CentOS 7.

Connecting to a Git Repo

You will probably want to connect to a git repository next. This is also somewhat dependent on the operating system you use, so I provide the steps to do this on CentOS as well:

  • Install git
sudo yum install git
  • Generate an SSH key on the server
ssh-keygen -t rsa
  • When prompted, save the SSH key under the following path (I got this idea from reading the comments here)
/var/lib/jenkins/.ssh
  • Assure that the .ssh directory is owned by the Jenkins user:
sudo chown -R jenkins:jenkins /var/lib/jenkins/.ssh
  • Copy the public generated key to your git server (or add it in the GitHub/BitBucket web interface)
  • Assure your git server is listed in the known_hosts file. In my case, since I am using BitBucket my /var/lib/jenkins/.ssh/known_hosts file contains something like the following
bitbucket.org,104.192.143.3 ssh-rsa [...]
  • You can now create a new project and use Git as the SCM. You don’t need to provide any git credentials. Jenkins pulls these automatically form the /var/lib/jenkins/.ssh directory. There are good instructions for this available here.

Connecting to GitHub

  • In the Jenkins web interface, click on Credentials and then select the Jenkins Global credentials. Add a credential for GitHub which includes your GitHub username and password.
  • In the Jenkins web interface, click on Manage Jenkins and then on Configure System. Then scroll down to GitHub and then under GitHub servers click the Advanced Button. Then click the button Manage additional GitHub actions.

additional actions

  • In the popup select Convert login and password to token and follow the prompts. This will result in a new credential having been created. Save and reload the page.
  • Now go back to the GitHub servers section and now click to add an additional server. As credential, select the credential which you have just selected.
  • In the Jenkins web interface, click on New Item and then select GitHub organisation and connect it to your user account.

Any of your GitHub projects will be automatically added to Jenkins, if they contain a Jenkinsfile. Here is an example.

Connect with BitBucket

  • First, you will need to install the BitBucket plugin.
  • After it is installed, create a normal git project.
  • Go to the Configuration for this project and select the following option:

BitBucket trigger

  • Log into BitBucket and create a webhook in the settings for your project pointing to your Jenkins server as follows: http://youserver.com/bitbucket-hook/ (note the slash at the end)

Testing a Java Project

Chances are high you would like to run tests against a Java project, in the following, some instructions to get that working:

Troubleshooting

  • If you cannot open the Jenkins web ui, you most likely have a problem with your firewall. Temporarily disable your firewall with: `sudo systemctl stop iptables` and see if it works then.
  • If it does, you might want to check your rules in `/etc/sysconfig/iptables` and assure that port 8080 is open
  • Check the log file at:
sudo cat /var/log/jenkins/jenkins.log

 

Continuous Integration Server Overview

Since I plan to set up a continuous integration server in the near future, I had a quick look around for open source and cloud-based solutions; my main concern was finding something which will work for a small scale project and result in reasonable costs.

Jenkins (Open Source)

The best choice if you are looking for an open source CI server. If you are familiar with Java, setting up and running Jenkins on your own is in all likeliness much cheaper than any cloud-based alternative.

Buildbot (Open Source)

Jenkins looks to be more widely used than Buildbot. However, if you have a Python project, Buildbot might be worth considering.

Travis CI (Cloud)

My top choice for open source projects. For commercial projects, however, the costs seem to be quite high starting with US$69 per month.

Circle CI (Cloud)

They offer one build container for free which seems like a very generous offer to me. I haven’t explored though how powerful this container is and how long builds would take.

AWS CodePipeline and AWS CodeDeploy (Cloud)

The best choice if you are using an AWS environment.

Codeship (Cloud)

They offer 100 builds per month for free which seems to be quite reasonable. However, since builds are triggered automatically this figure can be reached relatively quickly even with smaller projects.

 

Jenkins StackOverflow Exception during JUnit Test

Problem

The build of a Maven job in Jenkins fails due to a StackOverflowException or a OutOfMemoryException during a JUnit test.

Analysis

Most likely, there is a bug in your application, which leads to infinite recursion or an infinite allocation in resources. So first try to run your test in a different environment (for instance in eclipse, directly through Maven or as plain Java application).

However, sometimes, as in my case, the application works correctly and just requires more memory/stack size than is provided by the JVM during default settings. In this case, you will need to adjust the JVM settings in order to allow for more memory/stack size.

Solution

The JVM parameters, which allow to allocate more memory or stack size to a JVM are:

1. 

-Xss(x)m

where x is the size of the stack you would like to use. For instance: -Xss4m for a 4 MB stack size.

2. 

-Xmx(x)m

where x is the size of the Java heap space (=memory) you would like to use. For instance: -Xmx1024m for a 1024 MB heap space.

Unfortunately, there are multiple places in which these parameters can be configured. Essentially, if the increased heap space requirement is in your application, you will need to adjust the parameters for the JUnit test runner in Maven:


<plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-surefire-plugin</artifactId>
 <version>2.12.4</version>
 <configuration>
 <argLine>-Xss18m</argLine>
 </configuration>
 </plugin>

However, if the adjusted parameters are required for building your application, you might have to adjust them for the Maven Build Process.

This can be done by adding the parameters to MAVEN_OPTS either:

  • for your job: [Your Job] / Configure / Build / Advanced / MAVEN_OPTS
  • or for the Jenkins instance: Jenkins / Manage Jenkins / Configure System / Maven Project Configuration / Global MAVEN_OPTS

The last possible culprit might be the Jenkins server itself.

You can set the JVM parameters for Jenkins either by specifying them when:

  • starting Jenkins through java [your JVM parameters here] -jar jenkins.war or
  • by editing the jenkins.xml in your JENKINS_HOME folder.

Resources

Increasing Memory of junit Testcases

Stackoverflow – Jenkins build fails after running Cucumber tests on Java heap space exception

Increase heap size in java

Stackoverflow – How to give Jenkins more heap space when it´s started as a service under Windows?

 

Embedding Jenkins in Java App

The Continuous Integration server Jenkins can easily be deployed to any Servlet container or run in standalone mode (see Starting Jenkins).

However, sometimes it can be handy to be able to start Jenkins from within another Java application. I like this option since this ‘manager’ Java app can easily be deployed to other systems and platforms (without having to install Tomcat/Jetty etc). It is also useful to easily start Jenkins from an eclipse workspace (just run the Java file 🙂 ).

The only way I found to achieve this is to use Java to start anther OS process and then control Jenkins through this process.

Please find below example code to start a Jenkins process from a Java application. Don’t forget to change the paths and ports given as static variables in the beginning to your environment!


public class StartJenkins {

/**
 * Location of Jenkins executable
 */
 public static String JENKINS_WAR = "M:\\apps\\jenkins\\jenkins.war";

/**
 * Locations of Settings and Jobs
 */
 public static String JENKINS_HOME = "M:\\apps\\jenkins\\.jenkins";

/**
 * The Http port on which Jenkins web server shall be started.
 */
 public static String HTTP_PORT = "9090";

/**
 * The Https port on which Jenkins web server shall be started.
 */
 public static String HTTPS_PORT = "9999";

/**
 * AJP13 port for Jenkins.
 */
 public static String AJP13_PORT = "9003";

/**
 * @param args
 * @throws IOException
 */
 public static void main(final String[] args) throws IOException {
 final Process jenkins = startJenkinsProcess();

System.out.println("Press 'q' + enter to kill Jenkins");
 while (System.in.read() != Character.valueOf('q')) {
 System.out.println("Press 'q' + enter to kill Jenkins");
 }
 jenkins.destroy();
 System.out.println("Jenkins is done for!");
 }

public static Process startJenkinsProcess() {
 try {

final Runtime rt = Runtime.getRuntime();
 final Process proc = rt.exec("java -DJENKINS_HOME=" + JENKINS_HOME
 + " -jar " + JENKINS_WAR + " --httpPort=" + HTTP_PORT
 + " --httpsPort=" + HTTPS_PORT + " --ajp13Port="
 + AJP13_PORT);

final StreamRedirectorThread errorStreamProcessor = new StreamRedirectorThread(
 proc.getErrorStream(), "err", System.err);

final StreamRedirectorThread outputStreamProcessor = new StreamRedirectorThread(
 proc.getInputStream(), "out", System.out);

// start processors for output and error stream
 errorStreamProcessor.start();
 outputStreamProcessor.start();

return proc;

} catch (final Throwable t) {
 throw new RuntimeException(t);
 }

}

/**
 * based on <a
 * href="http://www.javaworld.com/jw-12-2000/jw-1229-traps.html?page=4">When
 * Runtime.exec() won't</a>
 *
 *
 */
 public static class StreamRedirectorThread extends Thread {
 InputStream is;
 String type;
 OutputStream redirect;

StreamRedirectorThread(final InputStream is, final String type,
 final OutputStream redirect) {
 this.is = is;
 this.type = type;
 this.redirect = redirect;
 }

@Override
 public void run() {
 try {
 PrintWriter redirectedWriter = null;

redirectedWriter = new PrintWriter(redirect);

final InputStreamReader isr = new InputStreamReader(is);
 final BufferedReader br = new BufferedReader(isr);
 String line = null;
 while ((line = br.readLine()) != null) {
 redirectedWriter.println(line);
 }

redirectedWriter.flush();

} catch (final IOException ioe) {
 ioe.printStackTrace();
 }
 }
 }

}