Remove Hard Disk in Linux in 3 Easy Steps

This guide describes how you can unlink a hard disk from Linux/Unix. This might be useful for instance if you replaced a disk image in Virtual Box or another VM.

WARNING: Do a backup of your virtual machine first or, if you are running on a physical computer, make sure you know what you are doing!

1. Assure the Hard Disk is not mounted

Edit /etc/fstab and assure there is no mount point for any partition of the hard drive.

IMPORTANT: Make sure that as many hard drives are identified by their UUID as possible since Hard Disk ids might change. See here.

2. Delete the Partition

Use fdisk as described here.

fdisk [your disk id eg /dev/sdb]

Note: You can find out the disk id by running fdisk -l (and use sudo if there is no output)

In fdisk running, input:

d

then input (assuming there is only one partition, otherwise give the number of a valid partition and repeat for all paritions):

w

3. Restart Machine

Shut down Linux

Disconnect your hard drive.

Restart Linux.

Your hard disk should be gone and no error should occur when you are staring.

Notes

When you are getting a message upon booting the machine that ‘The superblock could not be read or does not describe a correct ext2 filesystem.’ You are doing something wrong. Just reattach the hard disk in that case and Linux should start again. Make sure your other (not removed disks are identified by UUID as noted above).

 

 

 

Check if MySQL User Exists on Command Line (and in Puppet)

If you are using Linux, there is simple way to check if a user exists in MySQL:

echo `mysql -u root -p[your root password] -e ‘use mysql; SELECT user FROM user;’` | grep ‘[user name]

Executing this command will exit with return code 0 if the user exists and otherwise exit with 1.

This is very useful for building puppet scripts. The following execution will create a user if it doesn’t exist.

exec { “add user if not exist”:

unless => “echo `mysql -u root -p[psw] -e ‘use mysql; SELECT user FROM user;’` | grep ‘[username]‘”,
path => [“/bin”, “/usr/bin”],
command => “mysql -u root -p$mysql_password -e \”[Create User/Grant Rights Here]\””,
require => Service[“mysqld”],
}

Puppet Slow With Processing Directory with ‘recurse => true’

The Problem

Puppet allows deploying a structure of directories and files conveniently using the file resource.

Using the attributes ‘source => …’ and ‘recurse => true’, it is possible to create a copy of a set of files stored in the puppet module on target servers.

Unfortunately, using these attributes can slow down puppet significantly when the target directory on the server contains many files.

In my case, the time it took to apply a catalog jumped from under 5 seconds to more than 300 s.

The Solution

If you are lucky, the related puppet bug #9671 is closed in your version of puppet (not as of time of this writing).

If not, I found the following strategy to be effective:

  • Replicate your base directory structure using plain file resources (without recurse => true).
  • For subdirectories, do either:
    • Create a resource using ‘recurse => true’, for directories which contain many files/subdirectories in your puppet project.
    • Create more plain file resources for directories, which contain many files on the server.

Using this strategy, I was able to reduce the time to apply mentioned catalog below 10 s, while maintaining most of the convenience offered by the ‘recurse’ attribute.

Get Free Memory on Unix System in Java

Background

The opsunit framework allows to continuously run a number of JUnit unit tests and do some maintenance/repair work upon any failed test.

The latest addition to the test suite for the Appjangle platform is a simple unit test, which checks the remaining free memory on the server node. If the free memory goes beyond a certain threshold (e.g. 150 mb), some maintenance work is conducted in order to free memory on the node.

Since the server nodes are running a UNIX derivate, the Java unit test therefore needs to be able to determine the currently available system memory.

Problem

How to determine the available free system memory on a UNIX system from a Java program?

Solution

The first starting point is the ‘vmstat’ application. This application prints out all different kinds of information for the current system. Together with a few other GNU programs, it can be moulded to return the available free memory:

vmstat -s -S M | egrep -ie ‘memory|swap’ | grep ‘free memory’

It is possible to run this shell script from Java. There are multiple ways to achieve this, many of them troublesome and I ALWAYS do it wrong. Therefore I have assembled my hard-learned best practices (and those of others I could find) in the project java-start-process. Using this project, we can determine the free memory for a UNIX system using the following function:


public static int getFreeMemoryUnixInMb() {
 try {

final String expr = Spawn
 .runCommand(
 new String[] { "/bin/bash", "-c",
 "vmstat -s -S M | egrep -ie 'memory|swap' | grep 'free memory'" },
 null);
 // expr sth like " \t \t778 M free memory "

final String[] elems = expr.split(" |\t");
 for (final String elem : elems) {

try {
 return Integer.valueOf(elem);
 } catch (final Throwable t) {

}
 }

throw new RuntimeException(
 "Could not find free memory within: Elements="
 + Arrays.asList(elems) + " Raw Result=[" + expr
 + "]");

} catch (final Throwable t) {
 throw new RuntimeException(t);
 }
}

 

Resources

vmstat :: Detect Problems That Can Adversely Affect System Performance

Executing shell commands in java

executing shell script from java

Java Get Process Id: Three Approaches

Java provides no robust way to obtain the id of the OS process in which the application is running.

That does not mean, however, that there is no way. There are actually a couple of ways, each with its own advantages and disadvantages. I will describe three possible ways in the following:

1. ManagementFactory.getRuntimeMXBean().getName()

The class ManagementFactory in the package java.lang.management provides access to the “managed bean for the runtime system of the Java virtual machine”. The getName() method of this class is described as:

Returns the name representing the running Java virtual machine.

This name, as it happens, contains the process id in the Sun/Oracle JVM implementation of this methods in a format such as.


System.out.println(ManagementFactory.getRuntimeMXBean().getName());

// --> 742912@localhost

Through applying a simple String split, this thus allows to obtain the pid of the current process through the following expression:


ManagementFactory.getRuntimeMXBean().getName().split("@")[0]

Advantages:

  • Quick and dirty

Disadvantages:

  • Not guaranteed to work on all JVM implementations

2. Use Java Native Interface

At least on most UNIX based systems, it is relatively easy to write a quick JNA wrapper to the C library.


// Alternative 1: interface-mapped class, dynamically load the C library
public interface CLibrary extends Library {
 CLibrary INSTANCE = (CLibrary)Native.loadLibrary("c", CLibrary.class);
}

You can then add a mapping to the getpid function:


public interface CLibrary extends Library {
CLibrary INSTANCE = (CLibrary)Native.loadLibrary("c", CLibrary.class);

int getpid ();

}

And call this mapper as follows:


int mypid = CLibrary.INSTANCE.getpid();

Advantages:

  • Guaranteed to work as long as C library is available
  • Depends on JNA under com.sun. ….

Disadvantages:

  • Only works on UNIX based systems

3. Use the Java Virtual Machine Process Status Tool (jps)

If your JVM comes with the utility Java Virtual Machine Process Status Tool you can call this tool by spawning a new process, and analyse its output to obtain the id of your Java process.

Since there are a million ways to get starting a process from Java and reading its output wrong, I will use the utility java-start-process for this:


public static void getProcessId(final Class<?> mainClass,
 final Callback<String> callback) {

Spawn.startProcess("jps -l", null, new ProcessListener() {

@Override
 public void onProcessQuit(final int returnValue) {

}

@Override
 public void onOutputLine(final String line) {
 final String[] parts = line.split(" ");
 if (parts.length > 1 && parts[1].endsWith(mainClass.getName())) {
 callback.onDone(parts[0]);
 }
 }

@Override
 public void onErrorLine(final String line) {

}

@Override
 public void onError(final Throwable t) {

}
 });

}

Advantages:

  • Works on both Windows and Linux when Oracle/Sun Java is installed and on classpath

Disadvantages:

  • Complex
  • Requires Sun JVM

Resources

stackoverflow – How can a Java program get it’s own process ID?

stackoverflow – How do I obtain the PID of a spawned java process