Resize EC2 Volume (without Resizing Partition)

Problem

You would like to resize a volume attached to an EC2 instance.

Solution

Do the following:

  • Create a snapshot of your volume (instructions)
  • Stop your instance
  • Go to EBS / Volumes and select Actions / Modify Volume

Modify Vol

  • Enter the new size for your volume (note you can only ever make the volume larger) and click on Modify

size

  • Wait for the modification to be complete (this might take a while, like 30 min or so)
  • Start your instance

Now, if everything went well, you should have more space available on the disk for the virtual machine. To confirm this, run:

df -h

You should see the new size of the volume as the size of your main partition:

size2

Notes

  • If the size of your partition, does not match the size of the volume, you probably need to resize your partition (instructions).
  • Resizing the partition is a very painful process, that I think should best be avoided at all costs. I think for this it helps if the EC2 instance attached to the volume is stopped when the resize is performed. Assure that this is the case before you do the resize.
  • If you forgot to stop your instance, and need to do a partition resize, there is a little workaround. Wait for six hours, then resize your volume again (this time while the instance is stopped). Then, it hopefully adjusts your partition size to the correct size.
  • In the above, you might be able to start up your instance even while the new volume is still optimizing. I haven’t tested this though but my guess is that it would work.

 

Library for Parsing multipart File Upload with Java

One of the most convinient ways to upload files from the Web Browser to the server is by using file inputs in HTML forms.

Many web servers come with preconfigured modules for parsing this data on the server-side. However, sometimes, your HTTP server of choice might not offer such a module and you are left with the task of parsing the data the browser submits to the server yourself.

I specifically encountered this problem when working with a Netty-based server.

The form will most likely submit the files to your server as part of a multipart/form-data request. These are not that straightforward to parse. Thankfully, there is the library Apache Commons FileUpload which can be used for this purpose.

Unfortunately, processing some arbitrary binary data with this library is not very straightforward. This has motivated me to write a small library – delight-fileupload –  which wraps Commons FileUpload and makes parsing multipart form data a breeze. (This library is part of the Java Delight Suite).

Just include the library and let it parse your data as follows:

FileItemIterator iterator = FileUpload.parse(data, contentType);

Where data is a binary array of the data you received from the client and contentType is the content type send via HTTP header.

Then you can iterate through all the files submitted in the form as follows:

while (iter.hasNext()) {
 FileItemStream item = iter.next();
 if (item.isFormField()) {
   ... some fields in the form
 } else {
   InputStream stream = item.openStream();
   // work with uploaded file data by processing stream ...
 }
}

You can find the library on GitHub. It is on Maven Central. Just add the following dependency to your Java, Scala etc. application and you are good to go:

<dependency>
 <groupId>org.javadelight</groupId>
 <artifactId>delight-fileupload</artifactId>
 <version>0.0.3</version>
</dependency>

You can also check for the newest version on the JCenter repostiory.

I hope this is helpful. If you have any comments or suggestions, leave a comment here or raise an issue on the javadelight-fileupload GitHub project.

 

 

Set up MySQL Replication with Amazon RDS

Problem

You have an existing server that runs a MySQL database (either on EC2 or not) and you would like to replicate this server with a Amazon RDS MySQL instance.

After you follow the instructions from Amazon, your slave reports the IO status:

Slave_IO_State: Connecting to master

… and the replication does not work.

Solution

AWS provides very good documentation on how to set up the replication: Replication with a MySQL or MariaDB Instance Running External to Amazon RDS.

Follow the steps there but be aware of the following pitfall:

In step 6 `create a user that will be used for replication`: It says you should create a user for the domain ‘mydomain.com’. That will in all likelihood not work. Instead, try to find out the IP address of the Amazon RDS instance that should be the replication slave.

One way to do this is as follows:

  • Create the ‘repl_user’ for the domain ‘%’, e.g.:
CREATE USER 'repl_user'@'%' IDENTIFIED BY '<password>';
  • Also do the grants for this user
GRANT REPLICATION CLIENT, REPLICATION SLAVE ON *.* TO 'repl_user'@'%' IDENTIFIED BY '<password>';
  • Open port 3306 on your server for any IP address.
  • Then the replication should work.
  • Go to your master and run the following command:
SHOW PROCESSLIST;
  • Find the process with the user repl_user and get the IP address from there. This is the IP address for your Amazon RDS slave server.
  • Delete the user ‘repl_user’@’%’ on the master
  • Create the user ‘repl_user’@'[IP address of slave]’ on the master
  • Modify your firewall of your master to only accept connections on port 3306 from the IP address of the slave.
  • Restart replication with
call mysql.rds_stop_replication;
call mysql.rds_start_replication;
  • And check the status with
show slave status\G

The slave IO status should now be “Waiting for master to send event”.

 

 

 

Upgrade MySQL 5.5 to 5.6 on EC2/CentOS/RHEL

Problem

You would like to upgrade MySQL 5.5 to MySQL 5.6 on an existing server that uses the YUM software package manager.

Solution

Just enter the following few simple commands and you should be good to go. But, please, do a thorough full backup of your system before you do the upgrade just in case.

[1] Create a MySQL dump from which you will load the data into the upgraded server:

mysqldump -u root -p –add-drop-table –routines –events –all-databases –force > data-for-upgrade.sql

[2] Stop your MySQL server

sudo service mysqld stop

[3] Remove MySQL 5.5

yum remove mysql55-server mysql55-libs mysql55-devel mysql55-bench mysql55

[4] Clear the MySQL data directory

sudo rm -r /var/lib/mysql/*

[5] Install MySQL 5.6

sudo yum install mysql56 mysql56-devel mysql56-server mysql56-libs

[6] Start MySQL server

sudo service mysqld start

[7] Set the root password

/usr/libexec/mysql56/mysqladmin -u root password ‘xxx’

[8] Import your data

mysql -u root -p –force < data-for-upgrade.sql

[9] Verify all tables will work in 5.6

sudo mysql_upgrade -u root -p –force

All done!

Notes

  • Upgrade to 5.7 should work in a similar way, once 5.7 is available on your RPM repos (it isn’t by the time of the writing for the Amazon Linux Repo).

Sources

 

 

 

 

Fix Travis CI Error ‘This is not an active repository’

Problem

Your repositories have been building just fine using the tool Travis CI but suddenly the builds do not work anymore and the Travis CI website shows a screen with the message:

`This is not an active repository`

active

Solution

  • Go to GitHub and assure that you are logged in with the account that owns the repository.
  • Go to Travis CI and sign in with your GitHub account
  • Go to the repository
  • Click on the button ‘Active Repository’

If all works, that’s fine. However, if you get an error: ‘There was an error while trying to activate the repository.’ do the following:

  • Go to the settings for your account on Travis – Assure that the repository you want to build is enabled.

References

Travis CI Issue #5629

StackOverflow `Seeing “This is not an active repository” for an active repository`

Cannot Install VirtualBox Guest Additions: Installer Hangs

Problem

For a CentOS based guest, the update of the VirtualBox Guest Additions hangs after the step: “Removing exiting VirtualBox non-DKMS kernel modules”

install

Solutions

  • Check the log file in /var/log/VBoxGuestAdditions-uninstall.log. See if there is anything obvious reported there you can fix.
  • Check if your RAM disks are valid by running the command `sudo lsinitrd`.
    • If they are not, try rebuilding broken images with `sudo dracut -f [broken-image].img [kernel version]`
  • Verify you have all dependencies installed:
    • yum install dkms
    • yum groupinstall “Development Tools”
    • yum install kernel-devel
  • Reinstall your Kernel image
  • Verify that you have the correct version of kernel-devel installed
    • yum remove kernel-devel
    • yum install kernel-devel-`uname -r`
    • Restart system!
  • Wait!
    • Let the installation run for a while, at least 15-20 min. It might do something!

Java: Find all instances of System.out.println()

A good Java application should print only the absolute necessary to standard out. Not to do so can mean a serious hit in performance and can make it difficult to debug issues.

Unfortunately either we ourselves or our colleagues put System.out.println() statements in the code and then forget to remove them.

Here are two solutions how to find all those nasty statements.

Solution 1: Do a Full Text Search

Use your IDE and search for all occurrences of the string “System.out.println()” in your code. In eclipse this can be done through Menu / Search / File …

Make sure to use *.java for the file name pattern.

search

Solution 2: Put a Breakpoint in PrintStream

The above solution might not be practical if there are many System.out statements in the code which are legitimately there. In that case, you can also put a break point into the Java standard class PrintStream.

In eclipse, open the search for class dialog (Ctrl+Shift+t) and type in the class name PrintStream from the package java.io.

printstream.PNG

In this class, find the print(x) method and add a breakpoint:

breakpoint

Then run your program in debug mode. You can then find the places where System.out is called from the stack trace.