Tag Archives: open source

Free Tool for Renaming Files with Current Date

For many files, it is good to keep track of when they were originally created. In theory, every file created on any mainstream operating system should have its ‘creation date’ and ‘last date modified’ recorded as part of its metadata. This mechanisms, however, is often unreliable, especially when using file synchronisation tools such as Dropbox or Google Drive.

Thus it is often handy to prefix file names with the date they were created, such as:

2018 09 30 Letter.pdf

There are some nice tools available for this purpose, for instance Bulk Rename Utility. However, I often find these a bit too complex for what I need.

I have thus developed a little tool – Date Namer – , which does just the one thing I require: To prefix file names with the current date.

date-namer.PNG

Be welcome to download this tool from here:

Upcoming release will be published on this page: Releases. Further, all the source code for this tool is available on GitHub.

For those interested in the implementation details: For this project, I tried using Electron. This allows developing a Desktop application using Node.js. I found this overall quite easy to use. Internally this application will run an instance of Chromium to render the application. The running application takes thus around 50 MB of RAM. I think this is not too bad for this use case. The app performance is very good.

PlantUML (Open Source Awesomeness)

I’ve always had a soft spot for diagrams. I think that representing information in various visual ways tremendously helps our thinking and understanding. Unfortunately it is often a big headache to create (and maintain) diagrams.

So I was very pleased today when I came across PlantUML. PlantUML is a Java library and web service which renders UML diagrams from text input. Take the following text definition for example:


@startuml
object Object01
object Object02
object Object03
object Object04
object Object05
object Object06
object Object07
object Object08

Object01 <|-- Object02
Object03 *-- Object04
Object05 o-- "4" Object06
Object07 .. Object08 : some labels
@enduml

This will be rendered into the following diagram:

diagram

PlantUML does not just support object diagrams but also many other types of diagrams. There is another service, called WebSequenceDiagrams which focusses on only sequence diagrams (and is not open source) but can be useful if more visually pleasing sequence diagrams are required,

Setting Up Graylog Server

I have been looking around for an easy to use and reasonable priced solution for managing logs distributed among many servers and system metrics for these servers. I had a brief look into setting up an ELK system but I found that looked quite cumbersome. Recently I came across Graylog and I found it looked quite promising. I thus set up a little sample system.

While the documentation for Graylog is generally quite good, I found it a bit difficult to piece together the various steps in setting up a minimal working system. Thus I have documented these steps below!

Installing Graylog and Dependencies

Just follow the excellent CentOS installation instructions from the Graylog documetation.

Make sure to provide details for sending emails under the header # Email transport.

If you are using a firewall, open ports 9000 for TCP and 51400 for UPD. For instance, by assuring the following lines are in /etc/sysconfig/iptables.


-A INPUT -p tcp -m state --state NEW -m tcp --dport 9000 -j ACCEPT
-A INPUT -p udp -m state --state NEW -m udp --dport 51400 -j ACCEPT

Don’t forget to restart the iptables service: sudo systemctl restart iptables.

Collecting the Logs from Another CentOS System

  • Install rsyslog on the system

sudo yum install rsyslog
  • Enable and start rsyslog service (also see this guide)

sudo systemctl enable rsyslog

sudo systemctl start rsyslog
  • Edit the file /etc/rsyslog.conf and put the following line at the end, into the section marked as # ### begin forwarding rule ### (replace yourserver.com with your graylog server address.

*.* @yourserver.com:51400;RSYSLOG_SyslogProtocol23Format
  • Restart rsyslog

sudo systemctl restart rsyslog

The rsyslog log messages should now be getting send to your server. Give it a few minutes if you don’t see the messages in graylog immediately. Otherwise, check the system log for any errors (sudo cat /var/log/messages).

Also, you can test the connection by entering the following on the monitored system:


nc -u yourserver.com 51400
Hi

This should result in the message Hi being received by graylog.

Analysing Logs

The next steps are quite easy to to since they can be done in the excellent graylog user interface.

critical errors

  • Create an alert. Trigger it when there is ‘more than 0’ messages in the stream you have just created.

Done! You are now collecting logs from a server and you will receive an email notification whenever there is a serious issue reported on the server!

Continuous Integration Server Overview

Since I plan to set up a continuous integration server in the near future, I had a quick look around for open source and cloud-based solutions; my main concern was finding something which will work for a small scale project and result in reasonable costs.

Jenkins (Open Source)

The best choice if you are looking for an open source CI server. If you are familiar with Java, setting up and running Jenkins on your own is in all likeliness much cheaper than any cloud-based alternative.

Buildbot (Open Source)

Jenkins looks to be more widely used than Buildbot. However, if you have a Python project, Buildbot might be worth considering.

Travis CI (Cloud)

My top choice for open source projects. For commercial projects, however, the costs seem to be quite high starting with US$69 per month.

Circle CI (Cloud)

They offer one build container for free which seems like a very generous offer to me. I haven’t explored though how powerful this container is and how long builds would take.

AWS CodePipeline and AWS CodeDeploy (Cloud)

The best choice if you are using an AWS environment.

Codeship (Cloud)

They offer 100 builds per month for free which seems to be quite reasonable. However, since builds are triggered automatically this figure can be reached relatively quickly even with smaller projects.

 

Library for Parsing multipart File Upload with Java

One of the most convinient ways to upload files from the Web Browser to the server is by using file inputs in HTML forms.

Many web servers come with preconfigured modules for parsing this data on the server-side. However, sometimes, your HTTP server of choice might not offer such a module and you are left with the task of parsing the data the browser submits to the server yourself.

I specifically encountered this problem when working with a Netty-based server.

The form will most likely submit the files to your server as part of a multipart/form-data request. These are not that straightforward to parse. Thankfully, there is the library Apache Commons FileUpload which can be used for this purpose.

Unfortunately, processing some arbitrary binary data with this library is not very straightforward. This has motivated me to write a small library – delight-fileupload –  which wraps Commons FileUpload and makes parsing multipart form data a breeze. (This library is part of the Java Delight Suite).

Just include the library and let it parse your data as follows:

FileItemIterator iterator = FileUpload.parse(data, contentType);

Where data is a binary array of the data you received from the client and contentType is the content type send via HTTP header.

Then you can iterate through all the files submitted in the form as follows:

while (iter.hasNext()) {
 FileItemStream item = iter.next();
 if (item.isFormField()) {
   ... some fields in the form
 } else {
   InputStream stream = item.openStream();
   // work with uploaded file data by processing stream ...
 }
}

You can find the library on GitHub. It is on Maven Central. Just add the following dependency to your Java, Scala etc. application and you are good to go:

<dependency>
 <groupId>org.javadelight</groupId>
 <artifactId>delight-fileupload</artifactId>
 <version>0.0.3</version>
</dependency>

You can also check for the newest version on the JCenter repostiory.

I hope this is helpful. If you have any comments or suggestions, leave a comment here or raise an issue on the javadelight-fileupload GitHub project.