Go to content Go to menu

JFluidDB

12 October 2009

FluidEiffle If you read this blog regularly you’ll already know about FluidDB from FluidInfo. Well, I’ve written a Java based client library for it (I started from the excellent work by Ross Jones).

“FluidDB..?”

In a nutshell, FluidDB is a wonderfully simple yet powerful web based database that lives “in the cloud”. Objects exist in the database (there is only one instance of FluidDB), users tag objects and (optionally) associate values with the tags. A tag’s “value” can be anything: a number, a string of characters or even something more exotic like a picture, document or sound recording. Tags are organised with “namespaces” – hierarchies for organizing names – and can be searched using a minimalist query language.

“So far, so simple…”

Now here’s the interesting bit: all objects are public and never get deleted; it’s the namespaces, tags and their associated values that are covered by an elegantly simple permission system. FluidDB is open to write but powerful enough to enforce privacy.

“So what?”

I’d like to draw your attention to the adjectives I use in relation to FluidDB: “Simple”, “Elegant”, “Powerful”, “Minimalist”, “Open”.

If you’re a developer you’ll know that one of the most difficult aspects of software development is getting the abstraction and conceptual framework right. For example, you might think you’re reading a “document” in a “browser” on your “desktop” but these are just convenient names we use to make sense of our interactions with the computer. Such abstractions are layered from the lowest (1s and 0s) to the highest level that everyday users mostly see (“document”, “browser”, “desktop” etc). Abstractions in these layers of code are often (although not exclusively) expressed through an API (“Application Programming Interface” for you non-programmers, the public face a piece of software shows in order to interact with the outside world).

It is FluidDB’s API to which I refer when using the adjectives listed above. Any developer can create a badly designed API with lots of “features” and confusing terminology but it takes a rare discerning discipline to keep things as simple as possible but not too simple to be useless (to paraphrase Albert Einstein).

It is upon this API that I have written a Java library to allow developers to easily interact with FluidDB. JFluidDB is an evening and weekend project and my first stab at working with Java so all feedback is most welcome – especially if you find un-idiomatic Java. Below is an example that encapsulates all you need to know in order to work with it and FluidDB:

import java.io.IOException;
import com.fluidinfo.*;
import com.fluidinfo.fom.*;
import com.fluidinfo.fom.Object;
import org.json.*;
public class fluidTest {
    /**
     * Some example code for using the Fluid Object Model (FOM) classes with 
     * FluidDB
     * @throws JSONException 
     * @throws IOException 
     * @throws FluidException 
     * @throws FOMException 
     */
    public static void main(String[] args) throws FOMException, FluidException, IOException, JSONException {
        // The FluidDB class represents the instance of FluidDB you're connecting to.
        // The default constructor is set to use http://fluiddb.fluidinfo.com/ but we're
        // passing the URI to the sandbox here.
        FluidDB fdb = new FluidDB(FluidConnector.SandboxURL);
        // Login to FluidDB with your credentials
        String username = "username";
        String password = "password";
        fdb.Login(username, password);
        // Get the User object representing me
        User u = fdb.getLoggedInUser();
        // My root namespace
        Namespace root = u.RootNamespace();
        // Create a new namespace underneath my root namespace (name, description)
        Namespace books = root.createNamespace("books", "For storing tags about books I might be reading.");
        // Add some tags to the new namespace (name, description, indexed)
        Tag title = books.createTag("Title", "The title of a book I've read", true);
        Tag authors = books.createTag("Authors", "The author list", true);
        Tag hasRead = books.createTag("HasRead", "Indicates I've read this book", true);
        Tag rating = books.createTag("Rating", "Marks out of ten", true);
        Tag comment = books.createTag("Comment", "Some notes and commentary", false);
        // Create a new object in FluidDB (the argument is the "fluiddb/about" tag)
        // An object can represent ANYTHING
        Object seven_pillars = fdb.createObject("ISBN:0954641809");
        // Associate some tag/values with it
        // The first tag is only associating a tag but NOT a value
        seven_pillars.tag(hasRead);
        // We're associating values with these tags
        seven_pillars.tag(title, "Seven Pillars of Wisdom");
        seven_pillars.tag(authors, new String[]{"T.E.Lawrence"});
        seven_pillars.tag(rating, 8);
        seven_pillars.tag(comment, "The dreamers of the day are dangerous men, for they may act out their dreams with open eyes, to make it possible.");
        // A search of all objects that I have read (returns matching object's unique IDs)
        String[] result = fdb.searchObjects("has "+username+"/books/HasRead");
        // result will contain only one result... the id for the seven_pillars Object
        // Lets instantiate it and get a list of the available tags I have permission to see
        Object newObj = fdb.getObject(result[0]);
        String[] tagPaths = newObj.getTagPaths();
        // tagPaths will include my tags I created above...
        // Lets get the first tag and find out what is in it...
        Tag newTag = fdb.getTag(tagPaths[0]);
        FluidResponse r = newObj.getTagValue(newTag);
        // Assuming all is well the result is returned by calling r.getResponseContent(); 
        // Lets set / get some permissions
        // This will only give the current user and the fluiddb "superuser" account the ability
        // to CREATE namespaces underneath the namespace "book".
        // Permissions can have either OPEN or CLOSED policy and a list of exceptions, so this
        // Permission is CLOSED to everyone but fluiddb and username
        Permission p = new Permission(Policy.CLOSED, new String[]{"fluidDB", username});
        books.setPermission(Namespace.Actions.CREATE, p);
        // Lets get the permission policy for our user for updating the "books/rating"
        // tag
        Permission updateTag = rating.getTagPermission(Tag.TagActions.UPDATE);
        // Calling GetPolicy() and GetExceptions() on an instance of the Permission class will
        // tell you what the permissions are (as described in the FluidDB docs)
        // Lets do some cleanup...
        // Remove the tags from the object that we created to represent the seven pillars
        newObj.deleteTag(title);
        newObj.deleteTag(authors);
        newObj.deleteTag(hasRead);
        newObj.deleteTag(rating);
        newObj.deleteTag(comment);
        // Now delete the tags
        title.delete();
        authors.delete();
        hasRead.delete();
        rating.delete();
        comment.delete();
        // and finally the namespace
        books.delete();
    }
}

“But Nicholas, you’re a Python/.NET guy! Why Java?”

Good question! I have helped out with one of the early Python libraries and have written a functional .NET library (which will be brought up-to-date with JFluidDB), but I went for Java because of Android (Java is the development language for such mobile devices).

It strikes me that FluidDB would be an excellent database for lots of mobile applications and a good excuse to get into this field.

For example:

  • One could use the excellent ZXing barcode reader to scan objects (books?) in the real world to read and add tags (ratings, reviews, notes etc).
  • Should an object in FluidDB be tagged with a geo-location I should be able to display it on Google Maps or even an Augmented Reality viewer such as Wikitude (see my GeoCache project for an example of what this might be like).
  • I should be able to use my mobile device’s GPS, microphone and camera to add geo-location information, recordings, pictures and video to an object in FluidDB.
  • As FluidDB can easily be adapted to be a social network I’ll be able to use it just like one might use a mobile Twitter or Facebook client.

So, my next evening and weekend project is to create some simple Android applications as described above.

I’ll let you know when there’s something to show…

Network Graph

Yesterday evening we held our first code dojo kindly hosted by Fry-IT (who also supplied the pizza and beer). About 20-30 attendees turned up. Our aim was to generate a directed graph of a social network obtained from Twitter (something I blogged about before). You can see how we did by visiting the GitHub repository for the code dojo, the result of which is the image at the top of this blog post.

So, how did it go? Well, on the whole I got the impression that people enjoyed themselves but that there are many ways we can improve the “format”. Tim Golden’s blog post about the event pretty much sums it up.

For what it’s worth, here are my observations:

  • Most people were engaged most of the time during the dojo. Contrast this to developer conference talks where (to my eyes) most people seem to be staring at their laptops rather than the speaker (and I’m as guilty of this as anybody else).
  • People demonstrably enjoyed themselves: I especially liked the spontaneous rounds of applause when a unit test passed or a graph appeared.
  • It was friendly: suggestions were shouted out from the “floor” at appropriate moments but those in the hot-seats were also given space and silence when they just needed to get on with it.
  • The ten minute / pass the test rule (whichever came first) worked well at keeping the participants engaged and focussed and also meant there was regularly something / someone different to look at.
  • The accidental “half-time” break was a great innovation. As a group we were able to discuss the progress so far, what needed to be done next and what to clean up before hopping back into the chain of pair-programming.
  • Using a third party API (Twitter) and providing a code-scaffold was a mistake. The first pair spent a lot of time trying to understand code rather than write it. We’ve agreed to start completely from scratch next time.
  • Don’t use a Mac keyboard or limit the editors to VIM / Emacs. Far too many people were publicly put in an unfamiliar development environment. This was both unfair and unhelpful. We’ve agreed to use a “real” PC based keyboard and use a more “visual” editor (that doesn’t involve remembering key-strokes) next time.
  • Make the problem smaller and specific. Too many times during yesterday’s task it wasn’t immediately clear what should happen next because the overall task was relatively vague: “display a graph using Graphviz based upon social network data from Twitter”.
  • Try to make those in the hot-seats as comfortable as possible. To paraphrase one participant, “I sat down at the keyboard and my mind emptied of all programming knowledge”. The dojo is supposed to be a positive experience. Even typing in public can be nerve-racking.

Happily, most of the attendees said they would come to a second code dojo. So, in light of the list above, here’s what we agreed last night:

  • The next code dojo will take place on Thursday 15th October from 6:30pm-9:30pm-ish at Fry-IT’s offices (the same location).
  • Once again, there will be free pizza and beer (thanks Marcus!).
  • The task will be to create from scratch a Tic-tac-toe game that includes a simple AI as an opponent.
  • There will be some clear “baby-steps” and related tests to start the evening’s activity off with focus and direction. (I’ll supply these on GitHub).
  • We’ll do a post-mortem in the pub afterwards…

Get your thinking caps on for future code dojos: tasks, format changes, suggestions and ideas are all most welcome. I like the idea someone suggested of a team based dojo, reminding me of a Scrapheap Challenge like competition but for developers (imagine pitching that to Channel 4).

Finally, what does a neat table of pizza and beer look like after being attacked by Python hackers…?

Beer and pizza
(Image source)

Code Dojo

8 September 2009

Dojo On Thursday 17th September at 6pm there will be a Pythonic code dojo event taking place at Fry-IT’s offices.

So what on earth is a code dojo?

You could follow the link (above) to the nascent global code dojo site but, in summary, it is public collaborative coding with the aim of mutual learning based upon the premise that acquiring coding skill should be a continuous process. It seems that the original code dojo started in Paris sometime in 2005. (‘Dojo’ is a Japanese term roughly translated as “training hall” – often used when referring to martial arts or Zen Buddhism.)

A coding dojo will focus on a “kata” (another Japanese term meaning “form”). There can be two types of meeting:

  • Prepared Kata – where a pre-agreed presenter demonstrates how to solve a specific challenge from scratch using test driven development (TDD) and baby-steps. Each step must be simple enough to be understood by everyone and the presenter should only be interrupted should this requirement not be met.
  • Randori Kata – where a challenge is set and solved by pair programming (driver and co-pilot). Each pair has a small amount of time to advance the solution using TDD. When the time is up the driver goes back to the audience, the co-pilot becomes driver and one of the audience step up to be co-pilot.

We’re attempting a Randori Kata based meeting, but as none of us has ever organised a code dojo before, anything could happen (which is both a good and bad thing). I suspect the first part of the meeting will be to agree the ground rules followed the dojo itself.

We met a fortnight ago to agree a Kata and chose to create a pretty social network graph from Twitter data and Graphviz. I’ve built the “scaffolding” (available on GitHub) so we can jump right into the problem rather than worry about boilerplate code.

It’d be great to see you there…

FluidDB 101

19 August 2009

On Monday Terry and Esteve “switched on” the alpha version of their FluidDB offering. Congratulations guys!

I’ve written about FluidDB before – I find the philosophy behind the project so interesting. Now that I (and you) can get to play with it I’m recording some quick take-aways for the sake of my memory and hopefully your education (I’m assuming you’re au fait with FluidDB). ;-)

Currently, FluidDB is in read-only mode as none of the pillaging horde that are the alpha testers have been given access to their user’s details so can’t make any “writes”. According to Terry, alpha testers will get their account details by the end of the week.

Nevertheless, we can still “read” what little is in there to get a feel of what interacting with FluidDB is like:

Example Interactions

FluidDB currently communicates with the outside world via an HTTP based API. You could use tools like wget or curl to interact with FluidDB but simple libraries are already being written (kudos to Sanghyeon Seo for the quick work – my fork includes fixes for Python < 2.6). I’ll use my fork in the following Python doctest like example:

Return a list of objects that have the tag “username” from the “fluiddb/users” namespace


>>> import fluiddb
>>> fluiddb.call('GET', '/objects', query='has fluiddb/users/username')
(200, {'ids': ['8b57277a-09f6-485d-9108-761b7848c913', ...SNIP... '62fe2cca-7ad2-4dd8-9b0a-b3909c0709e8']})

Return an object where the tag “username” from the “fluiddb/users” namespace has the value “ntoll”


>>> fluiddb.call('GET', '/objects', query='fluiddb/users/username = "ntoll"')
(200, {'ids': ['5873e7cc-2a4a-44f7-a00e-7cebf92a7332']})

Find out about a specific object:


>>> fluiddb.call('GET', '/objects/5873e7cc-2a4a-44f7-a00e-7cebf92a7332', {"showAbout": True})
(200, {'about': 'Object for the user named ntoll', 'tagPaths': ['fluiddb/about', 'fluiddb/users/name', 'fluiddb/users/username']})

Get the value of the tag “fluiddb/users/username” from the object with the uuid “5873e7cc-2a4a-44f7-a00e-7cebf92a7332”


>>> fluiddb.call('GET', '/objects/5873e7cc-2a4a-44f7-a00e-7cebf92a7332/fluiddb/users/name')
(200, 'ntoll')

Get the result as json (GET and PUT responses default to raw HTML payload)


>>> fluiddb.call('GET', '/objects/5873e7cc-2a4a-44f7-a00e-7cebf92a7332/fluiddb/users/name', body=None, format='json')
(200, {'value': 'ntoll'})

So far so good…

Until we get our API keys that allow us to “write” stuff in our namespaces then this is all we’ve got to play with. I suspect that we might be able to create new objects (I’ve not tested this yet) as these don’t “belong” to anyone – remember it’s the namespaces/tags and the associated values that make FluidDB so interesting.

Finally, you might be asking where one might find out more… the most useful pages I’ve found are:

You might also want to join the #fluiddb channel on Freenode IRC or subscribe to the two Google groups.

I’ll update with more as soon as I’ve worked it out / been given access… :-)

Hello Android!

5 August 2009

Android

Last month I bought my first new mobile phone since 2001 (honest).

As a developer I wanted something I could play with, so after looking at the iPhone, Blackberry and Windows based offerings I ended up going for the HTC Magic running Google’s Android platform.

So far, I’ve found it to be a fantastic phone: easy to use, lots of very cool features and small enough to fit in your pocket. It’s definitely on a par with Apple’s OS (I own an iPod Touch) – and with other Android devices coming to market it looks like Google is onto a winner (again).

There are several fantastic applications available for Android phones. Two of may favourites are GeoBeagle and Wikitude – World Browser (both available in the Marketplace).

GeoBeagle

GeoBeagle is an open-source Geocaching application. What is Geocaching..? As the website explains:

“Geocaching is a high-tech treasure hunting game played throughout the world by adventure seekers equipped with GPS devices. The basic idea is to locate hidden containers, called geocaches, outdoors and then share your experiences online. Geocaching is enjoyed by people from all age groups, with a strong sense of community and support for the environment.”

As Android phones come with GPS (Global Positioning System) built it – just like the SatNav in your car – they can be used to determine your location and how far away you are from a cache. GeoBeagle is an excellent application that makes this easy – and it’s what I use when I go geocaching with my kids.

Wikitude

Wikitude World Browser is an example of augmented reality. I’ll let their website explain,

”[It] presents the user with data about their surroundings, nearby landmarks, and other points of interest by overlaying information on the real-time camera view of a smart-phone.”

When colleagues have asked about my new phone this is the one application I always show quite simply because it raises such incredulity and positive feedback that it is hard not to feel like a time-traveller from the future demonstrating a gizmo from Star Trek.

Happily, Mobilizy, the creators of Wikitude, have released an API (currently in closed Beta) and I requested to get on board. My intention was to mashup GeoBeagle and Wikitude as a means of learning Android development.

Yesterday evening I started experimenting. This evening I finished it off and managed to find enough time before sunset to run to my nearest geocache for testing, the results of which can be seen in the video below:

What does this show..?

  1. The freedom to learn from open-source projects such as GeoBeagle is invaluable for a newcomer to a platform such as Android. Bottom line: being open is an immediate win.
  2. Wikitude is an amazingly simple API to use. As you’ll read below, the Mobilizy guys have made developer’s lives very easy – my only concern being that such a simple API will make customisation difficult (although this is only the very first Beta release – I’m mentioning this for the purposes of feedback and fully expect Wikitude to mature).
  3. The Android platform is incredibly easy to learn, use, develop against and deploy to.

How..?

My background in software development includes C/C++, C#/.NET and Python. Android applications are written in Java and deployed to a Dalvik virtual machine – a “sandbox” for running Java especially designed for mobile devices. As a result I didn’t know what to expect. I got from nothing to the content of the video above with the following steps:

Step 1

You’ll need to download and install the Android SDK. Full instructions can be found on Android’s developer site. This will probably involve installing and configuring the Eclipse IDE. Don’t worry, the instructions are very clear and I managed to do it… ;-) (Also, the Android developer site is excellent with lots of tutorials, documentation and videos for the hungry mind.)

Step 2

Once you have the basic development environment installed you’ll probably need to tweak it and check it with a simple “Hello World” application. In my case I needed to make sure that Eclipse was targeting Java 1.6 (I’m on OS X where Java 1.5 seemed to be the default choice). I bought an e-book version of Hello, Android and chugged through the opening chapters happy that things seemed to be building and working as expected. I also made sure I could deploy / develop against my “real life” mobile phone (rather than the emulator provided). This simply involved plugging it into the computer. You’ll need to go to Settings -> Applications on your phone and make sure “Unknown Sources” is checked and that the appropriate settings are ticked under “Development” (I have “USB debugging” and “Stay awake” both checked).

Step 3

Get hold of GeoBeagle. The website’s wiki has excellent instructions but you need to be aware of the following:

  • You’ll need to have Subversion installed in order to get the source.
  • Don’t use the “trunk” as suggested in the documents. Use the “sng” branch. Grab the code by issuing the following command:

svn checkout http://geobeagle.googlecode.com/svn/branches/sng GeoBeagle

  • Follow the instructions on the wiki but be aware that you’ll have to make sure the di, gen and src directories are all referenced in the project’s build path (Properties for GeoBeagle -> Java Build Path -> Source).
  • The same goes for when you set up the unit tests. I found that the Android 1.5 jar wasn’t referenced in the Libraries section of the unit test project’s Build Path configuration.
  • I could only get GeoBeagle to run on my phone, not in the emulator. I’m assuming the lack of appropriate hardware emulation (such as GPS) is the problem here. In any case, developing on a “real” phone is a breeze.

Step 4

Sign up for the Wikitude Beta programme and wait for the code and documentation to arrive via email. To get Wikitude and GeoBeagle to play nicely together I simply did the following:

  • Reference wikitudearintent.jar in the Libraries section of the Java build path for GeoBeagle.
  • Add an appropriate button (I put mine in the menu for the cache list) and join up the event handling code.
  • In the event handling code do something like this:

// create the intent
WikitudeARIntent intent = new WikitudeARIntent(myActivity.getApplication(), null, null);
// add the POIs (points of interest)
Collection<WikitudePOI> pois = new ArrayList<WikitudePOI>();
for(GeocacheVector gv : this.mGeocacheVectors.getGeocacheVectorsList()){
        Geocache geocache = gv.getGeocache();
        float longitude = (float) geocache.getLongitude();
        float latitude = (float) geocache.getLatitude();
        String name = (String)geocache.getIdAndName();
        String description = "A description - grab info from geocache instance";
        WikitudePOI poi = new WikitudePOI(latitude, longitude, 0, name, description, null, null);
        pois.add(poi);
    }
    intent.addPOIs(pois);
}
// Add a title
intent.addTitleText("Augmented Reality View");
// Launch the intent
myActivity.startActivity(intent);

There are only two types of object a developer need worry about:

  1. WikitudeARIntent – the “intent” for the Wikitude World Browser (in Android an “intent” represents something that does a specific sort of thing – like display information using augmented reality as in this case).
  2. WikitudePOI – a Point Of Interest to display with the Wikitude intent. You supply information such as the longitude / latitude, altitude, name, description and so on.

I’m sure the code example above and description of the two classes is more than enough to see how the API works. Even the Wikitude documentation is only three pages long (as it doesn’t need to be any longer).

As I mentioned earlier, my main concern is that with such simplicity comes a lack of potential for customisation. For example, it would be great to override the menu that pops up if you double-tap a POI (as mistakenly happens in the video). Also, I’d like to be able to define area as well as individual points. Why? Consider the following use-case: wouldn’t it be great to be standing at the top of the Eiffel Tower and look down on Paris with all the various neighbourhoods highlighted and perhaps colour coded (indicating traffic congestion, for example)?

Wrapping Up…

Many thanks to Stephen Ng whose advice and patient help was most gratefully received when trying to get GeoBeagle to build. Without open-source developers like Stephen great tools like GeoBeagle would never exist. He’s worth his weight in gold! ;-)

Over the past two evenings I’ve had a lot of fun. I suppose the reason for this is both applications share the same outlook expressed in Wikitude’s tag-line:

“The World IS the Platform!”

...and who wouldn’t want to develop for that platform..?