New Repository for OSGi-ready Maven Artifacts

Although the onedb libraries are currently deployed as vanilla Java JAR files (onedb downloads), I have used the PDE tooling of eclipse heavily in the development process.

One core advantage of using OSGi in the development process is that OSGi allows to enforce information hiding between modules in a large Java application.

Unfortunately, most open source projects do not provide versions of their libraries, which are ready made for an OSGi environment. There is some great tooling available to create bundles from vanilla Java Jars, my favorite one being the Maven Bundle Plugin, which wraps the BND tool.

However, it can still be an annoyance to create your own OSGi-enabled maven artifacts for every dependency you want to use in your OSGi project.

This is my motivation to publish OSGi wrappers for all the third party dependencies I come across in my projects:

-> osgi-maven on github <-

Basically, these wrappers provided nothing more but a simple pom, which is pre-configured to generate valid OSGi declarations for the wrapped dependency:

OSGi Wrapper Artifacts

If the project is easy to build and/or there is no single artifact available, which could be wrapped, the wrapper might also contain the source code of the third party dependency. Therefore, please make sure that you use all wrappers in accordance with the source code license of the wrapped artifact.

Currently, the wrapper artifacts provide the following key features:

The following libraries are defined in the repository:

… and more coming.

Threads in GWT?

While it is widely reported that Google Web Toolkit does not support Java Threads and multi-threading, a number of aspects of concurrent applications can be emulated in GWT’s single thread JavaScript world.

The particular features that are relatively easy to emulate are:

  • Timers
  • Background Execution
  • Locks
  • ‘Thread-safe’ Collections

An important theme in my implementation of onedb was to write code in Java, which can be used both in a JVM environment and a GWT/JavaScript environment. Unfortunately, even though the Java Concurrency API and GWT’s concurrency features often provide similar features, their APIs are incompatible.

For instance, a codebase, which uses a java.util.Timer, cannot use a com.google.gwt.user.client.Timer at the same time.

This motivated me to write a simple abstract API (oneUtils), into which implementations for either a JVM environment or a GWT environment can be injected. Code, which uses this abstract API, can therewith be shared between JVM and GWT apps.

The abstract API currently supports the following features:

public interface Concurrency {

    public abstract TimerFactory newTimer();

    public abstract ExecutorFactory newExecutor();

    public abstract void runLater(Runnable runnable);

    public abstract OneLock newLock();

    public abstract CollectionFactory newCollection();

}

An implementation for JVM environments is included in the library (JreConcurrency), an implementation for a GWT environment is provided in the following gist:

Gist: Default Implementation of Concurrency API in GWT

Below a few usage examples for the API:

Usage of Executor API
Usage of Timer API
Usage of Thread-Safe Collections API

Please feel free to use the API + implementations. You can either grab the project from github or link to the project via Maven:

Dependency:

<dependency>
    <groupId>one.utils</groupId>
    <artifactId>oneUtils</artifactId>
    <version>0.0.3</version>
</dependency>

Repository:

<repositories>
    <repository>
        <id>onedb Releases</id>
        <url>http://dl.dropbox.com/u/957046/onedb/mvn-releases</url>
    </repository>
</repositories>

Eclipse and Github Tutorial

Github is an awesome repository to share your source code. Although there are numerous tutorials discussing how to use git and eclipse, I got stuck again today while trying to upload an existing eclipse project to github. This tutorial walks thus through all the steps from signing up for github to uploading an eclipse project to the site!

Please note that the focus of this tutorial is the mere upload of source code and not any of the more sophisticated features git and github offer.

The following steps will be discussed in this tutorial:

  1. Sign Up for github
  2. Installing EGit
  3. Create a DSA Key in Eclipse
  4. Register DSA Key with github
  5. Create Repository on github
  6. Import github Repository into eclipse
  7. Link Eclipse Project with github Repository
  8. Uploading Project Sources to github

Step 1: Sign Up for github

That’s the easiest part, just go to https://github.com/ and register!

Step 2: Installing EGit

You will need to install the git plugin for eclipse, EGit, in order to upload code from eclipse projects.

  • In eclipse, go to Help / Install New Software

  • Open the eclipse Indigo repository

  • Select Collaboration / Eclipse EGit

  • Click [Next]
  • JGit should have been selected for you automatically

  • Click [Next] and confirm the licence agreement
  • Restart eclipse and the EGit plugin should be installed

Step 3: Create a DSA Key in Eclipse

In order to be able to upload source code to github, you need to define a secure key, which must be known both to your local eclipse installation as well as the github service. Luckily, eclipse provides all the tooling necessary to generate the appropriate key.

  • Open Menu Window / Preferences
  • In the preferences, go to General / Network Connections / SSH2
  • On the SSH2 page, open the tab ‘Key Management’


  • Click on [Generate DSA Key …]
  • At the bottom of the page, enter a passphrase of your choosing


  • Click [Save Private Key …] (what’s going on with these three dots in the button captions … strange)
  • Save the key at a location of your choosing (best in the location specified as your SSH2 home on under the tab General)

Step 4: Register DSA Key with github

  • Open the file you have saved in the previous step with a text editor (e.g. Notepad on windows)
  • Select the contents of the file (Ctrl + A) and copy the complete text
  • Go to the github website (https://github.com) and login
  • On the top right of the screen, click on ‘Account Settings’

  • On the left hand side of the account settings, click on ‘SSH Keys’

  • Click on [Add SSH key]
  • Provide an appropriate title for your key (e.g. ‘EGit 1′ ?)
  • Paste the contents from the text file containing your DSA key into the text box ‘Key’

  • Click [Add Key] at the bottom of the form

Step 5: Create Repository on github

In order to upload source code from a project in eclipse to github, you will need to create a github repository.

  • Go to github homepage (https://github.com/) and log in
  • At the top right corner, click on ‘Create a New Repo’

 

  • Chose a repository name and description to your liking and click [Create Repository]

Step 6: Import github Repository into eclipse

Before you can link an existing eclipse project to a github repository, you must import the repository you have created on github first. For this:

  • In eclipse, open Menu / Window / Show View / Other …

  • Select Git / Git Repositories and click [Ok]

  • You might see a warning message such as the one show below (even setting the environment variable did not help me to get rid of the message, but everything seems to be working okay) – you can confirm with [Ok]

  • You should have a new view ‘Git Repositories now’
  • Click on ‘Clone a Git repository’ within the new view

  • Now go back to https://github.com and to your newly created github repository
  • Under your repository description, you can get the URI for your project. Copy the text starting with ‘git@’ (make sure that SSH is selected)


  • Go back to eclipse. You can paste the URI you have just copied into the field ‘URI’
  • Further select as Protocol ‘ssh’
  • Click [Finish]


  • If asked to select a branch, select the ‘master’ branch


Git (in difference to subversion) allows storing a full blown repository on your local machine rather than just a local copy of the files. This requires to store all source you want to synchronize with git at least twice on your local machine: one copy will be stored in the clone of the remote git repository and another one will be stored in your eclipse project.

Hence, when you close the git repository from github, you should define a repository location, which lies outside the eclipse project you want to upload:

  • Select one such location and click [Finish]


  • Now you should have one ‘Git Repository’


Step 7: Link Eclipse Project with github Repository

After you have created a local clone of the repository from github, you can link the eclipse project you would like to upload to this local repository.

  • Right click your eclipse project and select Team / Share Project …
  • Select ‘Git’ as Repository Type

  • Select under ‘Repository’ the repository you have cloned in the previous step and click [Finish]


 

Step 8: Uploading Project Sources to github

After you have linked your project with the local clone of the github repository, you can ‘Commit’ all the source files in your existing project to this repository. After you have committed the files to your local repository, you can ‘Push’ them to the github repository.

  • Right click your project and select Team / Commit … from the popup menu


  • Write a beautiful commit message and click [Commit]


  • In the ‘Git Repositories’ view, open <your repository> / Remotes / origin
  • Right click on the second icon (with the red arrow) and select ‘Push’


  • You should receive the results of the push, click [Ok] to confirm


  • You can now go to github and your project sources should be displayed in your repository:


(hmm, did it really took 2 hrs to get this done …)

References

Git with Eclipse (EGit) – Tutorial (vogolla.com)

Getting Started with Git, EGit, Eclipse, and GitHub: Version Control for R Projects

git push rejected (stackoverflow.com)

A Short Tutorial on Eclipse/EGit/GitHub

Tutorial: Using the EGit Eclipse Plugin with GitHub

 

 

onedb 0.0.3: Client API and Documentation Updated

A new version for the onedb client libraries (0.0.3) is now available for direct download or as maven dependency.

The new versions will work with any API key obtained from www.onedb.de.

onedb Java Test Toolkit:

    <dependency>
        <groupId>one.test.jre</groupId>
        <artifactId>oneTestJre.min</artifactId>
        <version>0.0.3</version>
    </dependency>

onedb Java Client:

    <dependency>
        <groupId>one.client.jre</groupId>
        <artifactId>oneClientJre.min</artifactId>
        <version>0.0.3</version>
    </dependency>

onedb release repository (in <project>):

    <repositories>
        <repository>
            <id>onedb Releases</id>
            <url>http://dl.dropbox.com/u/957046/onedb/mvn-releases</url>
        </repository>
    </repositories>

API Improvments

This release primarily implements feedback from Michael received by email (Thanks!).

Michael has pointed out:

The name of the ShutdownCallback class does not ‘fit in’ with the general naming pattern
for callbacks (When*).

This irregular naming pattern is of course no instance of a well-designed API! To remedy this incoherence, the ShutdownCallback has been renamed to WhenShutDown and the primary callback method has been renamed to the usual thenDo(...).

The select operation returns a list of strings when all children are selected but a list
of references in case the children are filtered according to some criteria.

The select operation returned a list of strings primarily for performance reasons. Thanks to some design decisions for engine and API, the list of children returned by the
select operation is a direct reference to the internal collection of ArrayLists, which is used as a local cache by the onedb client (wrapped, of course, in a unmodifiable list). This allows to access the list of child references with very little memory and CPU overhead.

However, premature optimization is the root of all … not good things.
Therefore, the default return value for the select operation has now been changed to a list of references, which is not only aligned with the other select operations but also allows to use the results of the select operation without having to wrap the returned string values in a One.reference(...) call.

Moreover, the option for good performance must not be abandoned. I have added a new operation variant One.select(...).allChildrenFast(), which allows to access a list of strings in the same way the One.select(..).allChildren() operation did in versions 0.0.1 and 0.0.2.

Documentation

All examples and tutorials have been updated to reflect the API changes in version 0.0.3.

A new document has been added to the documentation, which explains the various representations of nodes in onedb.

Various little changes have been made to the onedb tutorial to make some of the more complex issues easier to understand.

Bugfixes

The signature of the One.reference(..) was incorrect and did not allow to obtain the references of generic objects. The method signature has been fixed.

Compatibility

Clients built with version 0.0.1 and 0.0.2 will continue to work with the onedb cloud without need for modification.

A Practical Guide on Node Types in onedb

onedb is at its heart a cloud-based platform to connect pieces of information from various applications. These ‘pieces of information’ are represented and accessed in various ways in the onedb API.

The article “onedb Architecture and Design” provides a conceptual definition of most ways in which these ‘pieces of information’ are represented and the article “onedb Tutorial: Getting Started and First Steps” uses the various representations extensively.

In addition to the two articles mentioned above, this article provides a practical guide to understand and use the various data representations in onedb. In particular, the following four representations are discussed:

  1. Nodes
  2. Addresses and Identities
  3. References
  4. Value Nodes

1. Nodes

Nodes are the central component of onedb’s data model. Basically, every piece of information be it a basic data type such as String, Integer, etc. a custom object (new MyPerson()) or onedb object (One.newNode(..)) is managed as a node by the onedb engine.

However, only those objects are nodes, which have been appended and or loaded using the onedb API. For instance, in the following example text1 is not a node while text2 is:

String text1="I am just another string";
String text2="I will be a node";

One.append(text2).to(root).in(client);

Every node managed by a onedb client has got an address and an identity as described in the following section.

2. Addresses and Identities

Nodes in onedb are identified by a global unique identifier. These global unique identifiers are encoded as resolvable
Uniform Resource Identifiers (URIs).

Hence, every piece of information in onedb has a unique identity, which can be expressed in form of a resolvable address.

Examples for such resolvable addresses usually look as follows:


https://u1.linnk.it/4hxdr8/query/bob


https://u1.linnk.it/zednuw/types/customer


https://u1.linnk.it/4hxdr8/query/This_is_a_1

3. References

Most operations in the One API do not work with addresses directly but use so called references. A reference is nothing more than a simple wrapper object around an address.

onedb favors the usage of addresses ‘wrapped’ in reference objects over simply supplying an address in form of a String object to distinguish objects, which by chance hold a value which ‘looks like’ an URI, from objects, which are meant to represent links to entities within onedb.

Given an address in text form, it is very easy to create references…

OneTypedReference<?> reference1 = One.reference("https://u1.linnk.it/4hxdr8/query/bob");   
OneTypedReference<?> reference2 = One.newNode("https://u1.linnk.it/4hxdr8/query/bob").asReference();

// reference1.equals(reference2) == true

… and to obtain the address, which is wrapped by a reference object:

String address1 = reference1.getId();

// address1.equals("https://u1.linnk.it/4hxdr8/query/bob") == true

Moreover, for every object that has been added to a onedb client, a reference can be determined:

String text1="I will be a node";

One.append(text1).to(lr.root()).in(client);

OneTypedReference<Object> ref 
               = One.reference((Object) text1).in(client);

Note in the example above that the text1 object has been cast to Object in the invocation of the One.reference(..) operation. This is necessary for the API to know that no address but a generic object is passed to the operation. For other types of objects (e.g. Integer) this explicit cast to Object is not necessary.

References can also be used to obtain the resolved object of a node. For instance, the resolved object of the node text1 in the example above would be “I will be a node”. The resolved object of a node given its reference can be obtained as follows:

Integer value1=42;

One.append(value1).to(lr.root()).in(client);

OneTypedReference<Integer> ref 
                = One.reference(value1).in(client);

Integer value = One.dereference(ref).in(client);
// value.equals(42) == true

4. Value Nodes

While virtually any Java object can become a node (given the object’s class implements the Serializable interface), there is one special kind of object, which can be added to a onedb client: value nodes.

These objects are special in that they ‘know’ their own address. For instance, the object value1 above is of the class Integer and there is therefore no possibility this object would know its own address; since the class Integer is final in Java and no additional methods/attributes can be defined for instance of this class.

The interface OneNode is used to denote classes which have knowledge of their own address. This interface defines a single method getId(), which will return the address of the node/object. As it can be seen above, the mentioned reference objects belong to this type of objects.

Value nodes are a special type of object within the OneNode category. They are nodes, which implement the interface OneValue<Type extends Serializable> These nodes, apart from knowing their own address, define a value object of any type implementing the Serializable interface. This object can be accessed using the method getValue().

In the following example, a value node bob is created and its two properties address and decorated object are accessed:

OneValue<String> bob = 
        One.newNode("bob").at("https://u1.linnk.it/4hxdr8/query/bob");
bob.getId(); // == "https://u1.linnk.it/4hxdr8/query/bob"
bob.getValue(); // == "bob" 

Value nodes can turned into back and forth from reference to resolved object like any other object:

OneTypedReference<OneValue<String>> ref 
                             = One.reference(bob);

OneValue<String> value = One.dereference(ref).in(client);
value.getValue(); // == "bob"

Note that the call to One.reference(..) for the value node is made without the addition .in(client) as has been done above for the String and Integer objects. The client does not need to be specified for determining the reference of any object, which by itself ‘knows’ its own address.

It is usually a good practice to wrap objects whenever possible into value nodes. This allows for better performance, since the onedb engine does not have to derive the address of nodes from their object identity. Since object identities can also change in often unexpected ways, it is also usually safer to work with value objects.

The easiest way to define value nodes is generally by defining the .atAddress(...) parameter when appending a new object to an existing node. For instance, in the following example a new value node with the value alice is appended to the node root with the address ./alice relative to the root node’s address:

One.append("alice").to(root).atAddress("./alice").in(client);

Given the root node is defined at the address http://u1.linnk.it/example/root, this append statement would result in the creation of the following node arrangement:

One.value("root").at("http://u1.linnk.it/example/root")
--> One.value("alice").at("http://u1.linnk.it/example/root/alice")

Conclusion

This article has discussed the various ways in which data is represented and accessed using the onedb client. Essentially, all data is encoded in form of Java objects:

  • Any Java object can be turned into nodes by appending it to node in a onedb client.
  • Java objects, which are managed as nodes, have a globally unique address.
  • This address can be used to define a lightweight (and portable) reference to the Java object.
  • References can used to resolve the original defined Java object.
  • Java objects can be wrapped in special value nodes, which allow the onedb engine to manage these objects more effectively and securely.

onedb 0.0.2: Update for Client Libraries

I have just uploaded new onedb client libraries with version number 0.0.2.

The new client libraries mainly incorporate an advice from Java API expert Lukas Eder (check out JOOQ – a neat way to access SQL databases using a fluent Java API): All callback methods in the core API now have only one parameter; a simple data object, which allows access to the information, which before was handed to the callback method in form of parameters. For instance, the create realm operation in version 0.0.1 had the following primary callback method:

@Override
public void thenDo(OneClient client, OneNode realmRoot, String secret, String partnerSecret) { 
...

In version 0.0.2, the callback method for the create realm operation has been changed to contain only one parameter:

@Override
public void thenDo(WithRealmCreatedResult r) {
...

The individual parameters (client, realmRoot, secret, partnerSecret) can now be accessed through the WithRealmCreatedResult object:

client       : r.client()
realmRoot    : r.root()
secret       : r.secret()
partnerSecret: r.partnerSecret()

This has numerous advantages:

  • Since callback methods are defined very often in code using onedb, source code size is reduced: instead of listing all parameters and their types, only callback result objects have to be defined in the method signatures.
  • This reduction in source code, plus the avoidance of (unnecessary) type information increases the readability of the source code.
  • The API is much easier to change without breaking client code; since methods/data can be added to the *Result objects without having to change the callbacks in the client code.

The tutorial has been updated to reflect the changes of the revised API.

All clients using the API version 0.0.1 will continue to work.

onedb Tutorial: Getting Started and First Steps

onedb is the database engine at the heart of the appjangle platform. This tutorial gives an overview of a number of capabilities of the onedb engine.

The following topics are discussed in this tutorial:

Part I: Set Up

  1. Getting an API key
  2. Download the client library
  3. Linking the client library (eclipse or maven)
  4. Initializing onedb engine

Part II: Core Operations

  1. Create Realm
  2. Interlude: REST access using web browser
  3. Load
  4. Append
  5. Select
  6. Replace
  7. Remove

Part III: Advanced Operations

This part is still work in progress :)

  1. Commit (TBA)
  2. *Safe (TBA)
  3. Monitor (TBA)
  4. Post (TBA)
  5. Managing Authorizations (TBA)
  6. Clear Versions (TBA)

For further information about the motivation, concepts and design behind onedb, please check out the other articles on the onedb website.

Part I: Set up

1. Getting an API key

onedb is a cloud based database and therefore requires a cloud service provider to work.

Currently, onedb is deployed as part of the appjangle platform.

Just head to the appjangle page and sign up. You will receive an email with a key for appjangle. You can use this key to initialize the onedb engine.

2. Download the client library

Although onedb provides a simple REST interface, far superior usability and performance is provided by the supplied client libraries. You can download the latest version so the client libraries at

http://cms.onedb.de/downloads

Note that there are two different client libraries:

onedb Java Client: This client library provides all base features required to connect with the onedb cloud.

onedb Java Test Toolkit: Apart from all features of the onedb Java Client, the test toolkit includes a stripped down version of the onedb cloud. This allows to start up a local test cloud for unit tests (starting a test cloud should take less than 200ms).

Both libraries do not have any dependencies and are minified for reduced application size and optimal performance.

For this tutorial, pleas download the most recent version of onedb Java Test Toolkit from the downloads page. If you want to deploy an application to your users, please link the onedb Java Client instead.

3. Linking the client library (eclipse or maven)

The onedb client libraries can be linked to any Java application by adding them to the Java application’s classpath. Since this is rarely done manually, I will instead briefly describe how the library can be linked to an eclipse project or added to a Maven project. If you are already familiar with these procedures, feel free to skip to the next section.

Eclipse (Windows)

Link onedb to a new eclipse project by following the steps below:

  1. Download the onedb Java Test Toolkit library as discussed above and store the ‘oneTestJre.min-x.x.x.jar’ file on your local machine.
  2. You will need to download and extract a suitable eclipse distribution. The Eclipse IDE for Java Developers will do fine to begin with.
  3. Start eclipse and create a new Java project through Menu File / New / Java Project

Create a new eclipse project

  1. Provide a ‘Project name’ such as “exploreonedb” and press [Finish]
  2. Drag and drop the file ‘oneTestJre.min-x.x.x.jar’ onto the project you have just created in the eclipse ‘Package Explorer’

Drop JAR file

  1. Select ‘Copy files’ when prompted for a ‘File Operation’
  2. Expand the “exploreonedb” project on the eclipse workspace by clicking the small triangle left to the project name.
  3. Right click the file ‘oneTestJre.min-x.x.x.jar’ in the project folder and select Build Path / Add to Build Path in the context menu

Add jar to classpath

Now you can start using the onedb client library in the eclipse project. Go to step 4 to continue.

Maven

You can link the onedb client libraries to your Maven projects by using the following dependency declarations (in <dependencies>):

For onedb Java Test Toolkit

    <dependency>
        <groupId>one.test.jre</groupId>
        <artifactId>oneTestJre.min</artifactId>
        <version>x.x.x</version> <!-- replace with most recent version -->
    </dependency>

For onedb Java Client:

    <dependency>
        <groupId>one.client.jre</groupId>
        <artifactId>oneClientJre.min</artifactId>
        <version>x.x.x</version> <!-- replace with most recent version -->
    </dependency>

You will also need to add the onedb release repository (in <project>):

    <repositories>
        <repository>
            <id>onedb Releases</id>
            <url>http://dl.dropbox.com/u/957046/onedb/mvn-releases</url>
        </repository>
    </repositories>

4. Initializing the onedb engine

After adding the onedb libraries to a Java project, the onedb engine must be initialized. Adding the following statement anywhere in your application will initialize the onedb engine and prepare the engine to make request to the onedb cloud:

OneJre.init("[Your API Key]");

Please note that you will need to replace [Your API Key] with the API key you have obtained by email as described above.

If you have linked the onedb Java Test Toolkit, you can initialize the onedb engine as above (OneJre.init(..)) but also have the option to initialize the engine in test mode. The test toolkit can recreate a local version of the onedb cloud on a per test case basis. Just add the following statement instead of the statement listed above to any of your JUnit, TestNG, … test cases:

OneTestJre.init();

Note that since this will create a local onedb cloud, you will not need to supply an API key. Also note that the local onedb cloud will not respond to any REST requests for any created resources (for performance and portability reasons).

Part II: Core Operations

onedb uses only a handful of core operations to compose flexible and expressive data structures. The operations described in the following are a comprehensive list of the core operations supported by the onedb engine.

1. Create Realm

Realms are essential in working with onedb. Every item of data stored in the onedb cloud must be part of a realm. You can check the article “onedb Architecture and Design” to learn about the design concepts behind realms. This tutorial will walk through the practical steps necessary to create realms using the onedb Java API.

If you have followed the steps above to link the downloaded library in eclipse, you will have an empty eclipse project, including a link to the library ‘oneTestJre.min-x.x.x’. I will describe the following steps using eclipse in detail. If you are an eclipse whizz or use another IDE (or no IDE at all), please feel free to skip any of following steps.

You can add a Java file by right-clicking on the ‘src’ folder in your project and selecting New / Class. Add a new class ExploreOneDb as follows:

Create a new Java class

Add the statement OneJre.init(..) with your API key.

Initialize onedb engine

The main entry point to interact with the onedb client library is the class one.common.One. You can see all fundamental operations the onedb client library provides by typing One. and waiting for the context help to appear (below a screenshot for eclipse but the same will work in IntelliJ and NetBeans as well).

Context help for One class in eclipse

Chose the option createRealm(String withTitle):

Option Select realm selected

The onedb client now requires the specification of a title of the realm, which is to be created. Realm titles have no special significance apart from that they help to build descriptive URIs for any nodes stored in the realm. Chose the title "exploration" and add a . after the closing brackets to see the context options for the createRealm(..) operation:

Context options for Create Realm

We can either specify a client session by choosing the .in(OneClient client) option or specify the operation, we would like to perform after the realm has been created successfully by choosing the option .and(RealmCreated callback). Since there is no existing client session, we chose the second option.

The parameter type RealmCreated or more precisely When.RealmCreated will help to define a so called callback for the operation. A callback defines a method, which will be called once a remote operation is completed. Callbacks are used throughout the onedb APIs for all operations, which depend on remote systems.

Realms are managed by the onedb cloud and, therefore, the operation of requesting a realm needs to make a call to the remote onedb cloud. The duration of this call can vary significantly depending on your network connection and the way messages are sent through the Internet. In any case, the call to the remote cloud takes, in terms of computer time, very long. Using a callback enables your application to do some other useful work while it is waiting for the response from the onedb cloud.

To define the callback for the createRealm(..) operation, chose the option and(..) and type within the brackets after and: new When.. This will show the available callbacks in the onedb API. Select the callback definition, which corresponds to the required parameter type indicated on top of the parameter.

Select Callback

A bit of cleanup needs to be performed before proceeding: A semicolon needs to be added at the end of the (now completed) statement and the When class containing the callback definitions needs to be added to the imports.

Clearing Syntax Errors

The method thenDo(..) will be called ‘back’ upon successful creation of the new realm on the onedb cloud. If the parameter name for the thenDo(..) method are something as informative as arg0 .., we can rename the parameter with a more descriptive name such as result or r:

@Override
public void thenDo(WithRealmCreatedResult r) {

The result object will carry the following values:

r.client(): The createRealm operation will create a new client session, which can be accessed through the client parameter. This session will also have the realmRoot and its children available for further operations.

r.root(): The realmRoot parameter points to the root node of the newly create realm; to this node, new nodes to be stored as part of the realm can be appended.

r.secret(): The secret parameter holds a String, which must be supplied when the realmRoot is accessed using the onedb Java Client or the REST API (It’s a kind of access token).

r.partnerSecret(): This parameter will hold no value after the conducted invocation of the createRealm operation. This parameter will hold a secret, which will allow to write (but not read) a node for a postbox type realm.

While the application as given above will successfully create a realm, we will not be able to access this realm after the application is closed, since we will neither know the address of the realm nor the secret token to access it.

To save the realm’s address and access secret, we can, for now, print them to the console by adding a few print statements as follows:

@Override
public void thenDo(WithRealmCreatedResult r) {
    System.out.println("realmRoot: "+r.root());
    System.out.println("secret: "+r.secret());
}

Running the application should result in an output such as the following:

realmRoot: One.reference("https://u1.linnk.it/bkesvc/explora")
secret: maqi______z94

You will notice that applications starts, then prints the output above after a few seconds, but will not stop. In order to terminate our application correctly, we have to shut down every client session we have created. In our case, this is the one client session created by the createRealm operation.

To shut down the client session, we can add the following to the existing application after the last System.out statement:

...
System.out.println("secret: "+r.secret());

One.shutdown(r.client()).and(new When.Shutdown() {

    @Override
    public void thenDo() {
        System.out.println("Session is shut down.");
    }

});
...

When we start the application again, it should now terminate as expected after printing the access information for the realm.

realmRoot: One.reference("https://u1.linnk.it/bgbpce/explora")
secret: gv3______0nx
Session is shut down.

Save both the URL of the node (“…/explora”) and the printed secret for the next steps of the tutorial in a text or source file. You can also rerun the application at any time to obtain this information again.

Note that although we have not changed any parameters of the createRealm operation, the second invocation resulted in a different realm root and a different secret being reported.

[full source code of example on github]

2. Interlude: REST access using web browser

We can access the newly created realm(s) using any web browser using the REST API of onedb: just type in the URI of the realm root reported by your application (e.g. https://u1.linnk.it/bgbpce/explora) into the browsers address bar and hit enter.

If you are greeted by the following friendly message, just select [Proceed anyway] or the available equivalent of your browser.

Add SSL Exception

Supply as authentication the username ‘token’ along with the secret access token reported by your application (e.g. gv3etqoingxe0nx) and press login.

Http Basic authentication

Select save password if you are given the option. You should see the sparse contents of your node displayed by the web browser.

Node rendered as html

You can see different representations of your node by appending a variant (.node. or .value.) and a data format (.html, .xml or .json) to the URI of your node such as:

https://u1.linnk.it/bgbpce/explora.value.xml or

https://u1.linnk.it/bgbpce/explora.node.json

The variant value with the format XML, for instance, should be rendered as follows:

XML Value representation

3. Load

While it is convenient to access nodes using the provide REST interface, it is far easier to access the created nodes using the onedb API. As long as we know the access token (secret) and address of a node, we can access it from any system linked to the onedb library.

For the purposes of this tutorial, we can create another class in our Java application to emulate another app accessing the node. Add a class to your project ‘Load’ and add a main method to this class.

New Class Load

Although both ExploreOneDb and Load are part of the same eclipse project, they form two distinct applications for Java, since both classes have independent main methods. The onedb engine must be initialized once for every Java application. Since Load defines a new application, we need to initialize the onedb engine as follows:

OneJre.init("[your API key here]");

Next we type One. once again but this time chose the operation load(Object node):

Select Load Operation

The load operation now requires the specification of an Object node. This object specifies what is to be loaded. The only information we have available from the invocation of the ExploreOnedb example app is the address of the root node of the realm (e.g. https://u1.linnk.it/bgbpce/explora) plus its access secret you have saved (if not, just rerun the ExploreOnedb application).

onedb distinguishes between resolved nodes with a value and references to nodes. Node references have a unique identity but have no value. We can use such a reference to specify, which node we want to load, for instance:

One.reference("https://u1.linnk.it/bgbpce/explora");
                        // ^-- replace with your URI

Replace the address in above statement with the address you have saved after running the ExploreOnedb example and provide the reference definition as the node to be loaded for the load operation. Again type a . after the closing bracket to see the further parameters for the load operation. Select the parameter .withSecret(String secret):

Parameters for Load Operation

Supply the access token secret you saved earlier (e.g. “gv3__0nx”). Finally, you should define the callback by selecting the option .and(...):

Specifying Callback Parameter

Proceed in the same way as specifying the callback for the createRealm(..) operation, but this time chose the callback new When.Loaded(..). Your application should now look as follows:

OneJre.init("[your API key here]");

One.load(One.reference("https://u1.linnk.it/bgbpce/explora"))
                               // ^-- replace
   .withSecret("gv3______nx")
   .and(new When.Loaded() {

        @Override
        public void thenDo(WithLoadResult<Object> lr) {

        }
});

The callback method (thenDo(..)) returns a load result with a reference to the node, we have just loaded (lr.loadedNode()). The load result also returns a client session (lr.client()). We can retrieve the resolved loaded node as follows:

@Override
public void thenDo(WithLoadResult<Object> lr) {
    Object realmRoot = One.dereference(lr.loadedNode()).in(
            lr.client());

    System.out.println("Node reference: " + lr.loadedNode());
    System.out.println("Resolved node: " + realmRoot);
}

For an in-depth discussion of the various node types and the One.dereference(..) operation please check the article “A Practical Guide on Node Types in onedb“.

Run the application and you should get an output such as the following:

Node reference: One.reference("https://u1.linnk.it/bgbpce/explora")
Resolved node: Nx.define(exploration).at(https://u1.linnk.it/bgbpce/explora)

[full source code of example on github]

4. Append

Thus far, this tutorial walked through the steps of setting up a Java project, creating a realm and accessing this realm using a REST interface and the onedb API. Although creating a realm inevitably results in the creation of one node (the realm root), we have not really done a lot of work with nodes, for instance establishing connections. In this section, I will explain the most important operation to define nodes and connections between them: append.

First create a new class NodeOperations and add a main method, which creates a new realm (check the creating a realm section above for details regarding the following code snippet):

public static void main(String[] args) {
    OneJre.init("[Your API Key]");

    One.createRealm("ops").and(new When.RealmCreated() {

        @Override
        public void thenDo(WithRealmCreatedResult r) {

        }
    });
}

Let’s assume the following scenario: We would like to define a customer in the newly created name with the name “Bob” who lives at “26 Short Av”. To express this information using a proven and tested object-oriented approach should not be too difficult: First, we define a class Customer and subsequently create an instance for Bob.

Class definition:

public static class Customer {
    public String name;
    public String address;
}

Instance Creation:

Customer bob = new Customer();
bob.name = "Bob";
bob.address = "26 Short Av";

We need to make one small modification before we can upload the Bob object to the onedb cloud: The customer class will need to implement the Serializable interface, in order for onedb to be able to transport objects of this type safely to the onedb cloud. This is easy enough; just change the class definition to:

public static class Customer implements Serializable {
    public String name;
    public String address;
}

We can now append the bob object to the ‘ops’ realm root node as follows:

@Override
public void thenDo(WithRealmCreatedResult r) {
    Customer bob = new Customer();
    bob.name = "Bob";
    bob.address = "26 Short Av";

    One.append(bob).to(r.root()).in(r.client());

    System.out.println("Created " + r.root() + ":" + r.secret());
    // do shutdown ...
}

[full source code on github]

You can run the application and should receive the login information for the test realm such as Created One.reference("https://u1.linnk.it/crd87h/ops"):hhz______ni. We can use this reference to load both the realm and the associated bob object using the One API as described above under the load operation.

We can also access the newly created realm using a web browser as described in the REST section above.

REST Access to realm

The picture above points to a problem with the approach taken so far: Although the customer has been added to the realm as a SerializedNode (class NodeOperations$Customer), the data associated with this node cannot easily be interpreted through the REST interface. Indeed, also other Java applications would have difficulties in ‘deciphering’ the data of this node without the definition of the class Customer in byte code. This stays in contrast with the ideals of a small data system!

A better approach is to decompose the information a Customer instance expresses into various nodes with ‘standard’ data types (such as String, Integer …). One way to do this could be as follows:

The root node of the realm is designated to represent the bob entity. We append a node with the text “Bob” to this realm root node to indicate the name. We also append another node with the text “26 Short Av” to the realm root. To indicate the ‘type’ of the used nodes, we further append two generic nodes Address and Customer, which do not hold any particular value to “26 Short St” and the realm root respectively.

The described arrangement of nodes can be visualized as follows:

Example Scenario

We can create a new class NodeOperationsBetter and again specify the logic for creating a new realm. The following code snippet shows how to define a node arrangement as described above using the onedb API.

OneJre.init("[Your API Key here]");

One.createRealm("ops").and(new When.RealmCreated() {

    @Override
    public void thenDo(WithRealmCreatedResult r) {
        OneClient client = r.client();
        Object bob = r.root();

        One.append("Bob").to(bob).in(client);

        String addressValue = "26 Short Av";
        One.append(addressValue).to(bob).in(client);
        One.append("an Address").to(addressValue).in(client);

        One.append("a Customer").to(bob).in(client);

        System.out.println("Created " + r.root() + ":" + r.secret());
        // shutdown client ...
    }
});

[full source code on github]

If we access the realm created with the logic above using the REST interface, the data should be presented in a more accessible manner such as below:

Screenshot: Bob Node

The REST interface allows navigating from one node to another. If we click on ’26 Short Av’, the node representing the address value will be displayed:

Screenshot: Street Value Node

Apart from presenting the information associated with the customer in a more accessible manner, following a connection-oriented approach makes the data semantically richer. For instance, we will have implicitly created a globally accessible type ‘address’ (linked to 26 Short Av Node). The type URI for address will look something like the following:

https://u1.linnk.it/l8hpud/ops/26_Short_A2/an_Address0

However, this URI does not appear to be very ‘pretty’ and portable. In specific, the part ’26ShortA2′ does appear to be in conflict with the intention to define a reusable identity for the type ‘address’.

It is usually a good practice to define ‘type’ nodes in their own independent realm. This way their reusability can be increased and also shorter and more succinct URIs can be created. Type nodes can be appended to a realm like any other node. We can write a little application such as the following to define the type nodes required for the example:

One.createRealm("types").and(new When.RealmCreated() {

    @Override
    public void thenDo(WithRealmCreatedResult r) {
        Object typesRoot = r.root();

        Object addressType = One.append("an Address").to(typesRoot)
                .atAddress("./address").in(r.client());

        Object customerType = One.append("a Customer").to(typesRoot)
                .atAddress("./customer").in(r.client());

        System.out.println("Address type: " + addressType);
        System.out.println("Customer type: " + customerType);
        System.out.println("Types realm " + r.root() + ":" + r.secret());
        // shutdown ...
    }
});

[full source code on github]

Note here the slightly changed append statements with an added atAddress(..) parameter. Specifying the atAddress parameter allows to specify a precise address to be used for nodes; if atAddress is not specified, onedb will attempt to generate a suitable address.

Running the above application should result in an output such as the following:

Created One.reference("https://u1.linnk.it/zednuw/types"):ip_______ib5
Address type: One.value(an Address).at("https://u1.linnk.it/zednuw/types/address")
Customer type: One.value(a Customer).at("https://u1.linnk.it/zednuw/types/customer")

Save the output of your application in a text file or somewhere else for the next steps.

Using the newly created ‘type’ nodes, we can rewrite the definition of customer bob as follows (please remember to change the type nodes in the examples to the type nodes you have created):

// -- reference types
Object addressType =
  One.reference("https://u1.linnk.it/zednuw/types/address");
                             // ^-- replace with your type!
Object customerType =
  One.reference("https://u1.linnk.it/zednuw/types/customer")
                             // ^-- replace with your type!
// -- build data
OneClient client = r.client();
Object bob = r.root();
One.append(customerType).to(bob).in(client);
One.append("Bob").to(bob).in(client);

String addressValue = "26 Short Av";
One.append(addressValue).to(bob).in(client);
One.append(addressType).to(addressValue).in(client);

[full source code on github]

Now, if we were to define a customer Alice, we could reuse the type nodes for ‘address’ and ‘customer’ used for the definition of Bob.

5. Select

Querying data in onedb is done on a recursive from-node-to-node basis much in the spirit of Linked Data. In order to aid the navigation from node to node, onedb provides three operations to query the children of a node:

selectFrom(node).allChildren(): Will return a list of the references of all children appended to the node.

selectFrom(node).allChildrenFast(): Will return a list of addresses as Strings of all children appended to the node.

selectFrom(node).theChildren().withType(type): Will return a list of references to all children of the node with the specified (Java) type.

selectFrom(node).theChildren().linkingTo(reference): Will return a list of references to all children of the node, which have the specified reference as one of their children. For instance, in the example node arrangement given below, selecting all children from the node persons linking to Customer will return the nodes bob and alice.

Example Arrangement linkingTo

Create a new Java class with main method, initialize the onedb engine, create a realm and define the following nodes for the realm:

One.append("This is a test realm").to(r.root()).in(r.client());
Object bob = One.append("bob").to(r.root()).atAddress("./bob")
        .in(r.client());
One.append(
        One.reference("https://u1.linnk.it/zednuw/types/customer"))
                           // ^-- replace with your type node
        .to(bob).in(r.client());

[full source on github]

Note that you will have to replace the reference to the customer type defined above ("https://u1.linnk.it/zednuw/types/customer") with the customer type you have created in the previous section.

Run the application and you should save the link and access token for the realm you have created (e.g. One.reference("https://u1.linnk.it/4hxdr8/query"):zt________y2).

Then, create another Java class with main method, initialize the onedb engine and load the node you have just created using the link and secret you have saved:

One.load(One.reference("[your query realm]"))
        .withSecret("zta_____1y2")
        .and(new When.Loaded() {

            @Override
            public void thenDo(WithLoadResult<Object> lr) {
...

Note that you will have to replace the reference and supplied secret with the details of the query realm you have created above.

Within the thenDo(..) callback of the load operation, you can retrieve the references to all children of the loaded node:

System.out.println("All Children: "+
    One.selectFrom(lr.loadedNode()).allChildren().in(lr.client()));

You can filter the children of the loaded node by their type:

One.selectFrom(lr.loadedNode())
        .theChildren()
        .ofType(String.class)
        .in(lr.client())
        .and(new When.ChildrenSelected<OneTypedReference<String>>() {

            @Override
            public void thenDo(
                    WithChildrenSelectedResult<OneTypedReference<String>> sr) {
                System.out.println("Found Messages:");
                for (OneTypedReference<String> node : sr.children()) {
                    System.out.println("  "
                            + One.dereference(node).in(
                                    sr.client()));
                }
            }

        });

Note that the operation to select children by type in difference to the previous one (select all children) requires the specification of another callback (When.ChildrenSelected). As a general rule, onedb will always require the specification for all operations, which may need to send a remote message to the onedb cloud. The initial loading of the root node of the realm, will download the root node from the onedb cloud including the references to all its children. A reference, however, is not sufficient to determine the (Java) type of a node. Therefore, the select operation with type parameter will need to assure all child nodes have been downloaded from the onedb cloud.

The same applies for specifying the linkingTo(..) parameter. Since the load operation initially only loads the children of a node but not its children’s children, a remote request must possibly be sent to the onedb cloud, requiring the specification of a callback:

One.selectFrom(lr.loadedNode())
        .theChildren()
        .linkingTo(
                One.reference("https://u1.linnk.it/zednuw/types/customer"))
                                 // ^-- replace with your customer type
        .in(lr.client())
        .and(new When.ChildrenSelected<OneTypedReference<Object>>() {

            @Override
            public void thenDo(
                    WithChildrenSelectedResult<OneTypedReference<Object>> sr) {
                System.out.println("Found Customers:");

                for (OneTypedReference<Object> node : sr.children()) {
                    System.out.println("  "
                            + One.dereference(node).in(
                                    sr.client()));
                }
            }

        });

[full source code on github]

Running your application should result in an output such as shown below.

All Children: [One.reference("https://u1.linnk.it/4hxdr8/query/NxAuth.rea0"), One.reference("https://u1.linnk.it/4hxdr8/query/This_is_a_1"), One.reference("https://u1.linnk.it/4hxdr8/query/bob")]
Found Messages:
  This is a test realm
Found Customers:
  One.value(bob).at("https://u1.linnk.it/4hxdr8/query/bob")
All queries completed.

The select example makes heavy use of references (One.reference(...)) and value nodes (One.value(..).at(..). For an in-depth discussion of these different node types and how one node type can be converted into another, please check the article ‘A Practical Guide on Node Types in onedb‘.

6. Replace

Apart from creating realms and appending nodes in complex and deep arrangements, onedb supports to replace the value of nodes as well as removing nodes. However, it is often a good idea to avoid these operations whenever possible. onedb in its core is designed to enable distributed systems: any piece of data or node might be opened by multiple clients on the same or different devices.

The operations remove and replace add one significant factor of uncertainty: mutability. While immutability is a nice property of any software system, it is crucial to the success of distributed systems.

As long as we constrain ourselves to using the operations of createRealm, append and select, the data stored in onedb will be immutable. For instance, when we append a node representing the type customer to a ‘types’ node, it can be guaranteed that this node will be available to any client working with the system.

The operation remove, in particular, can lead to unexpected and undesired situations in a system with many involved clients. However, there are cases, where using update and remove is just by far the simplest solution. To support these cases, onedb offers operations both to update and remove nodes from the network.

The update operation will replace a node value with another node value. The connections of a node remain unaffected by the update operation. The value of nodes with externally managed address can easily be updated as shown in the following:

String phase1 = "phase1";
One.append(phase1).to(realmRoot).in(client);

Object phase2 = "phase2";
One.replace(phase1).with(phase2).in(client);

The example above will first add a node “phase1″ to the onedb cloud and then replace this node with a node with the value “phase2″. Please note that although the value of the node has been changed, the address of the node will stay the same:

Before update:

Value  : "phase1"
Address: https://u1.linnk.it/di14a2/update/phase11

After update:

Value  : "phase2"
Address: https://u1.linnk.it/di14a2/update/phase11

In general, it is not allowed to change the address of a node using the update operation. This can become tricky when working with nodes with internally managed addresses. For instance, the following operations present a BAD practice:

OneNode phase1Node = One.append("phase1").to(realmRoot)
        .atAddress("./p1").in(client);

One.replace(phase1Node)
        .with("phase2"))) // WRONG !!!
        .in(client);

The above example would replace a node "phase1" WITH a specified address "./p1" with a node value "phase2" WITHOUT a specified address.

The correct way to update a node with an internally managed address would be as follows:

OneNode phase1Node = One.append("phase1").to(realmRoot)
        .atAddress("./p1").in(client);

One.replace(phase1Node)
        .with(One.newNode("phase2").at(phase1Node.getId()))
        .in(client);

[full source on github]

onedb stores a version every time a node is changed (e.g. replaced, a child is appended, a child is removed ..). Therefore, calling One.clearVersions(...) for nodes, which are frequently changed, can significantly increase the performance of loading and manipulating data. You can find an example on github on how to use the clear versions operation.

7. Remove

As already mentioned in the previous section, replace and remove are operations, which should be used with care in environments, where one node is synchronized between multiple clients. Remove, in this regard, is more ‘dangerous’ than replace, since replace guarantees that once defined nodes are available to other parts of a distributed system. Remove, in contrast, can render nodes unavailable to other components of a system. Therefore, remove should always be used with caution when working with the onedb cloud!

The remove operation can basically be used for two purposes: First, removing one node from another node will delete the connection between these nodes (if a connection has been defined before). However, remove can also be used to remove a node from the onedb cloud; a node will be removed from the onedb cloud, if the connection to its direct parent is removed.

The following snippet will first append a node "to be removed" to the realm root and define it in the onedb cloud. Then, this node will be removed from the realm root AND the onedb cloud.

    // remove connection AND node
    OneNode toBeRemovedNode = One.append("to be removed")
            .to(realmRoot).atAddress("./toBeRemoved").in(client);
    One.remove(One.reference(toBeRemovedNode)).fromNode(realmRoot)
            .in(client);

If, however, a connection between nodes, which are in no direct parent-child relationship is removed, ONLY the connection will be removed and not the connected node. In the following example, the node "to be kept" will still be defined in the onedb cloud even after it has been removed from the node "another node".

    // remove ONLY connection
    OneNode toBeKeptNode = One.append("to be kept").to(realmRoot)
            .atAddress("./toBeKept").in(client);

    OneNode anotherNode = One.append("another node").to(realmRoot)
            .atAddress("./anotherNode").in(client);

    One.append(toBeKeptNode).to(anotherNode).in(client);
    One.remove(toBeKeptNode).fromNode(anotherNode).in(client);

[full source on github]