Automatically Make Snapshots for EC2

A quick Google search reveals that there are quite a few different approaches for automatically creating snapshots for EC2 images (such as herehere and here).

All of these are rather difficult to do.

Thankfully, after some more searching around I found a great way to schedule regular snapshots using AWS CloudWatch.

CloudWatch supports a built-in target for ‘Create a snapshot of an EBS volume’:

target

For details of how this can be set up, see the excellent step-by-step instructions on the CloudWatch Documentation.

Delete All Binary Logs for MySQL

Today I discovered that one of my servers mysteriously ran out of disk space.

I ran the following Linux command to find all the biggest files and folders on the server:

sudo du -a / | sort -n -r | head -n 100

… and found that it was the binary logs used for MySQL replication that were gobbling up all the disk space:

7375152	/
4691636	/var
4324880	/var/lib
4284952	/var/lib/mysql
1079420	/usr
1048588	/var/lib/mysql/mysql-bin.000004
1048584	/var/lib/mysql/mysql-bin.000006
1048584	/var/lib/mysql/mysql-bin.000003
802356	/var/lib/mysql/mysql-bin.000007

Now I first found some advise that using the PURGE BINARY LOGS should be the way to go. That is true if you want to delete the logs without hurting your ongoing MySQL replication.

However, I was just interested in deleting all the binary logs and the way to do that is by logging into your server with a user with SUPER privileges and executing the following command:

RESET SQL;

Now all those pesky ‘mysql-bin.*’ files should have disappeared!

 

Generate md5 Hash for Maven

Maven creates and checks MD5 checksums at various times. For instance, when downloading an artifact from a repository, Maven checks whether the checksum of the downloaded files (e.g. POM, JAR) is correct.

I am uploading Maven artifacts manually to a very simple, file-based repository. This requires me to calculate the Maven checksum for artifacts manually.

What sounds like a simple problem at first actually turned out to be quite tricky (as it often does with Maven? Well, I guess it’s still better than regular expressions!).

After digging around in the project checksum-maven-plugin, I finally figured out how to generate MD5 checksum files in a way that Maven will accept. The key here was to use the Hex class from Bouncy Castle to turn the MD5 digest into a String.

Following the rough-cut code to create a hash file for Maven (extracted from the maven-tools project, class WriteHashes):

public static void writeMd5(final Path baseFile) throws NoSuchAlgorithmException, IOException {
        final FileInputStream fis = new FileInputStream(baseFile.toFile());

        final MessageDigest messageDigest = MessageDigest.getInstance("MD5");
        messageDigest.reset();
        final byte[] buffer = new byte[1024];
        int size = fis.read(buffer, 0, 1024);
        while (size >= 0) {
            messageDigest.update(buffer, 0, size);
            size = fis.read(buffer, 0, 1024);
        }

        final String result = new String(Hex.encode(messageDigest.digest()));

        fis.close();

        final Path md5File = baseFile.getFileSystem().getPath(baseFile.toString() + ".md5");

        FilesJre.wrap(md5File.toFile()).setText(result);
    }