I must be on a roll - two blog posts in a matter of hours.
Like many who live in the UK I know Stephen Fry as a "celebrity" with a love for "gadgets". However, this recent interview demonstrates that he also groks software, design and usability.
My favourite section contrasts what he calls functional software with Apple's offerings (starting around 6 minutes in):
"[It's] as if these devices are only function objects, and that's what Apple realise, supremely, and others are now beginning to realise. The point is they're made for human beings and human beings are first and foremost emotional creatures. We are creatures of emotion. Our emotion hits the brain, this is study-able - you can see this on encephalographs and things. Emotions hit us before cognitive thought and that means that if we have an object that's in our pocket all the time [...] and we take it out it's something we have a relationship with: we touch it, we feel it, we look at it - whether or not we want [to], that means we have an emotional engagement. And therefore anybody who produces it merely to say, "press this it does this, press that it does that" doesn't understand what it is to be human.
Apple understand that we are all, as human beings, people who want to cradle, stroke, fondle, smile, get annoyed, treat "as alive" an object. That's not pretentious, it's not what Ruskin would call "the pathetic fallacy" it is the way all humans respond to what is around us. Apple got this by making us smile, by making us delight in the things they offered. And now, fortunately, all the other big players: Google and HTC and even Microsoft (god bless them), Motorola and Palm - they're all understanding that users want (to use that terribly hackneyed word) an "experience". Not just a series of functions that "this provides that" as if it's a cupboard, a filing system and that's it - they want to hug it and technology finally allows that."
I agree with him: good software puts the human user at the heart of what it does.
“I’ve only just discovered FluidDB and the reoccurring question is, ‘how is FluidDB different/better than CouchDB??’”
I’ll try to answer as best I can.
The usual caveat applies when using a loaded word like “better” – it depends on what you want/need. As I can’t speak for anyone else I’ll only do a compare and contrast.
Lets concentrate on the similarities first… both databases are flat collections of items (“documents” in CouchDB, “objects” in FluidDB) with a structure based upon the classic subject/predicate/value triple:
CouchDB: Documents have fields with values.
FluidDB: Objects are tagged (often with values)
Documents <=> Objects (both are identified by a UUID),
Fields <=> Tags (both are dynamically typed),
Values <=> Values
A simple difference in terminology.
(Actually, in FluidDB an object/tag pair does not need to have an associated value. Simply associating a tag with an object can provide enough information. For example, the object that represents the book Dune might have the “ntoll/has_read” tag associated with it and as I am the only person within FluidDB to have permission to associate this tag with objects you can infer from the name of the tag that I once read the referenced book. I’m getting ahead of myself here, but it’s important to point out that a tag-value is optional in FluidDB.)
The other obvious similarity is that both DBs have a RESTful API (hence the potential for CouchApps and Flow – web-applications as data hosted within their own database).
Now for the differences in no particular order:
There is only one FluidDB floating “up there” in the cloud and it holds everyone’s data. With CouchDB companies and individuals are responsible for hosting their own instance(s) and such instances are usually created for a specific purpose/application.
The predicate part of the triple is very different. Fields in CouchDB are simply names of values. In FluidDB tags can be organised with namespaces. You start with an empty root namespace named after your username and create tags and namespaces underneath this. Just to be clear here, you can’t associate an object with a tag that isn’t yet defined – and yes, tags and namespaces are also objects in FluidDB so you can meta-tag. ;-)
Querying data is very different. CouchDB’s “Views” are pre-defined map/reduce based algorithms. FluidDB provides an uber-minimalist (yet still evolving) SQL-like declarative language.
Security and permissions in CouchDB are document centric and define who can read data and what validation steps a user needs to pass before they can write data to a document (CouchDB also has the concept of an admin account). The “model of control” (as Terry Jones [creator of FluidDB] calls it) is FluidDB’s killer feature: permissions apply only to tags, namespaces and tag-values. All objects are public and can be tagged by anyone.
The implications of point four are not at first obvious but it is definitely this feature that sets FluidDB apart from CouchDB and every other database that I know of. Because of this I’ll spend the rest of this post illustrating why the model of control is so important.
Because all objects can be tagged by anyone lots of interesting information/behaviour begins to emerge and become possible.
Remember the “Dune” example? Because only I have permission to associate the “ntoll/has_read” tag with an object you can be pretty certain only those objects with this tag have been read by me. I can also control who sees my tags, their values and even allow other trusted users the ability to add, edit, and delete namespaces, tags and tag-values (in a similar way to being able to set permissions on a filesystem). This is important because it allows people to collaborate: users who are food enthusiasts could all agree to use the “foodies/rating” and “foodies/review” tags on objects representing restaurants. Only those users enthusiastic/trusted enough would be allowed into the group with permission to associate such tags. Furthermore, as “insiders” they might also have created a tag called “foodies/discount” that is only visible to them and, when attached to an object representing a restaurant, explains how to get a discount.
Even though some objects have an optional “fluiddb/about” tag (holding a unique value and set by the object’s creator to provide some guidance as to what the object is supposed to represent) the only way to find out what an object really represents is to look at what tags and values are associated with it. The tags and associated values cast an outline of a sort of “data-shadow” identifying the object’s referent.
For example, the object with the fluiddb/about tag-value “book:DUNE” might have the following subset of tags/values associated with it:
{
fluiddb/about: "book:DUNE",
ntoll/has_read,
tim_oreilly/has_read,
amazon.com/type: "Paperback Book",
amazon.com/title: "Dune",
amazon.com/author: "Frank Herbert",
amazon.com/isbn: "ISBN123456789",
amazon.com/price: "$9.99",
amazon.com/genre: "Sci-Fi",
amazon.com/cover: (BINARY DATA FOR A PNG FILE),
tom/comment: "I really like the Sandworms",
dick/opinion: "Far fetched and obtuse",
sally/rating: 5,
books/other_editions: ['UUID for object x', 'UUID for object y', 'UUID for object z'],
books/isbn: "ISBN 123456789",
books/title: "Dune",
books/author: ['UUID of object representing Frank Herbert']
books/publisher: "Chilton Books",
books/first_published: "1965"
...
}
Now, consider the following:
In the same way that #hashtags and @username conventions have emerged on Twitter so conventions such as “username/has_read” will emerge within FluidDB (for example). I use it because a well known publisher of tech books might use it, along with many other people. Put simply, the schema of FluidDB evolves.
Organisations that can verify they own a domain name can claim it as a root namespace within FluidDB. As a result Amazon has tagged the book with useful information.
Tag values do not have to be strings… they can be anything and the mime-type used when HTTP PUTting them into FluidDB is what is used when retrieving them. As a result, the amazon.com/cover value is an image/png file (presumably of the book’s cover). Actually there are two types of tag value: primitive and opaque – the difference is explained here (put simply, you can search against primitive values).
The tags from the “books” namespace provide lots of useful information, much of it the same as that provided by amazon.com.
The books/other_editions tag-value is a list of UUIDs identifying other objects related to this one in a similar way to how foreign keys are used to reference related tables/rows in relational databases.
Lots of different people, organisations and applications have added tags to the object.
I’ll never see some of the tags associated with this object because I don’t have permission to.
If Amazon.com decide to delete their tags from this object it is still useful and all the other data is unaffected.
Another side-effect of the FluidDB model of control is that completely unrelated applications will be able to share data. This is already happening: Terry has written an application called Tickery that has imported lots of information from Twitter under the “twitter.com” namespace in FluidDB. Because this data is open for everyone to read I can make use of it from within Flow and carry out exactly the same searches as Tickery does (e.g. “has twitter.com/friends/jack and has twitter.com/friends/ev”).
Suddenly the potential for mashing up data becomes huge and very interesting – especially as anyone can add further data to the objects Tickery and Flow have tagged.
In conclusion, we’re all familiar with social networks – FluidDB is simply a social database ‘in the cloud’ with its model of control as the secret magic sauce.
Since the end of last year I’ve been spending quite a lot of my spare time experimenting with FluidDB as a platform for web-development. The result is a still incomplete application that I call Flow (the source-code is hosted on GitHub).
Flow is a simple “show and tell” project to demonstrate how to write small javascript based applications for interacting with FluidDB. The aim is for Flow to become an application for writing javascript based applications with FluidDB.
I’ll let you pause a minute while you work out the circularity of that last sentence. ;-)
To make things clearer I’ve cobbled together a short screen-cast that attempts to demonstrate what I’ve been up to. (It’s very rough and ready, but you’ll definitely get the idea.)
It’s VERY incomplete, hardly tested properly, extemporised (rather than designed) and needs a lot of TLC – just a proof of concept really. Nevertheless, as the video demonstrated, you can:
Sign in/out
View, create, edit and delete namespaces/tags
Edit tag values for mime-types text/* in all browsers and the ability to do the same for image/* and application/pdf and application/vnd.ms-powerpoint values in Firefox version 3.0 and above (more on this later).
Search for objects using the simple query language that comes with FluidDB.
To use it properly you’ll need to register for a username and password here:
Lets just remind ourselves of how FluidDB is organised from the point of view of data / schema:
Objects represent things in FluidDB.
Users define attributes to attach to objects – these are tags that can be organised with namespaces.
When a tag/attribute is associated with an object it can (optionally) hold a value.
The value of a tag/object association can be anything but you can (optionally) specify a mime-type for each value.
This is all we need to understand for now. (FluidDB’s “killer feature” is actually its permissions system and the implications thereof. It is so important that I’ll save that topic for its own blog post later. But one step at a time…)
So how does FluidDB act as a web-server..?
Simple really (and apologies if this is too simple an explanation – I don’t like to assume too much technical expertise)...
Remember that FluidDB is RESTful in that its API is exposed through the HTTP protocol (y’know, that thing the World Wide Web is built out of). So, if I want to retrieve the value of a tag/object association I simply HTTP GET a URL – just like a web browser does for web-pages.
So how about I create an object to represent my web-application, add a bunch of namespaces and tags that map to the names of the files and directories needed for the application to work and then associate these tags with my application’s object? Of course, I make sure the tag/object values are the contents of the files and that the mime-types are set properly (e.g. values for tags mapping to .html files have the mime-type “text/html”).
This means that if a browser does an HTTP GET request to the path for a tag/object value within FluidDB and the mime-type for the value is set to “text/html” it’ll display it as a web-page. The same goes for javascript (application/x-javascript), CSS (text/css), images (image/*) and anything else (application/pdf for example).
Hey presto… FluidDB is a universal web-server.
Sammy.js
So we have a server for our application, but FluidDB doesn’t allow any server-side scripting. This means we need to put our application’s logic somewhere else – the only other place being on the client’s browser.
Happily, we can use a browser’s Javascript capabilities for this and doing so is not so much a leap as other well-known web-applications already do this (GMail, Google Docs, Zoho and so on).
What is more, the guys in the CouchDB project have been thinking along similar lines for a while and Aaron Quint has developed a javascript framework for browser-side web applications called Sammy.
Sammy is a tiny javascript framework built on top of jQuery. Because of a clever “hack” it is able to “trap” GET and POST requests from a page and handle them with javascript functions associated with “routes”. It’s certainly packed full of features for such a small application and I’ve been hacking away with a maniacal grin on my face as I’ve been exploring Aaron’s work.
If you’ve still not figured out why this is important, take a look at these two images Aaron has created (extracted from this presentation and used with permission) to show the difference between “regular” web-application development and that promoted by CouchDB and (now) FluidDB:
Sammy allows us to live in a Lisp like “code is data” world with no web-server / web-framework playing piggy-in-the-middle. The database is the server and the framework is Sammy-as-data running on the client.
HTTP PUT, Forms, Mozilla & Binary Data
In order to create or update a tag on an object the API requires a PUT request containing the appropriate value. Unfortunately, if the data we want to send is not text based we have a problem.
The usual way to upload binary data is to use an HTML input element with the “type” set to “file”. Unfortunately, forms can only use the POST and GET HTTP verbs (this seems an amazingly bad decision – surely they should only support POST and PUT..?).
Rather than change the whole Internet I investigated how to get at the raw data from the request with Javascript and re-frame it as a PUT request to FluidDB. It turns out this is impossible except for users of Firefox.
Since version 3.0 (or 1.9 of the Gecko engine that Firefox uses) it has been possible to do this with the getAsBinary() function. In fact Mozilla have also provided the rather useful getAsDataURL() and getAsText() functions too. These are definitely non-standard and more information about them can be found on Mozilla’s website.
You can see how I did it by looking at the “upload_to_mozilla” function in the tag_value_app.js file in the javascript directory/namespace.
Unfortunately, this means I’m stymied for the other browsers and I’m not sure of the best solution. Of course, extracting textual data from a form is easy on all browsers so this only affects binary data. Nevertheless, it’s a problem that probably should be solved if such database-hosted applications become popular.
That’s all (for now)...
I’ll keep plugging away at Flow and will post updates on Twitter.
I also have to give a presentation in March at ThoughtWorks in London about FluidDB. I was thinking… it might be fun to create a “presentation” application with FluidDB and Sammy. Something along the line of DOM slides, AJAX-S or S5.
Last night was the fifth London Python Code Dojo. At the end of the December dojo we agreed to try a different format: a well defined problem to be tackled concurrently by small teams who have a pre-defined amount of time to complete it in (about an hour). This would be followed by a show and tell / compare and contrast session.
We hadn’t decided on a problem before the dojo so we chose to follow a suggestion made at an earlier dojo. A white-board was provided for attendees to write up suggestions during the pizza / beer part at the start of the evening.
This seemed to have worked well and a consensus was reached to write an “old skool” style text-adventure game over the course of the next few dojos.
Why..?
Well, a classic text-based adventure game contains lots of opportunity to practice and experience different aspects of computer-science / programming. I can think of the following problems off the top of my head:
Client / Server / Multi-user support (a la MUD / MOO – perhaps using Twisted?)
Make a UI to display images and other multi-media
Last night we attempted to build and navigate a game world. The fruits of our efforts can be found here on GitHub.
Happily, the team based format worked exceptionally well affording more people the opportunity to write code and solve the problem. Furthermore, Dave Kirby kicked off the evening with a lightning talk about using CRC cards in small-group development teams – rather appropriate given the context. Thanks Dave!
The most entertaining part of the evening (for me) was the “show and tell” conclusion. Each team was given the use of the projector and five minutes to demonstrate the game, show their code (and tests!) and talk through their design and methodology. It was fascinating (and funny) to see how differently the teams had tackled the problem (four out of five had working examples) – yet there were also many similarities. Highlights included a solution created entirely through TDD, fancy-pants autocomplete courtesy of the CMD class, a solution that separated game data from the game engine and had its own (proprietary) file format and a game with some very imaginative writing, pirate voice overs and a very large octopus.
On Tuesday and Wednesday of this week I attended droidcon, the Android developer’s conference organised by the friendly folks at Skills Matter and Kevin and Carl from Novoda – the most enthusiastic mobile software development company on the planet.
Tuesday was a barcamp and a great opportunity to meet people and learn from others. I especially enjoyed Carl’s talk on his work using RESTful APIs from Android (something I’ve been experiment with in conjunction with FluidDB). Other highlights of the day included learning about the Zii Egg and recently announced Trinity device, seeing the worlds longest nested if/else statement (don’t ask – very diagonal) and some really great chats both technical and “blue sky” about the potential for mobile (and specifically Android) devices.
The second day was a more traditional presentation-based conference and we had some great talks. Highlights for me were a couple of presentations that delved into and demonstrated the NDK (native development kit), both of which were musical in nature (spot on for me) and had me dusting off my C/C++ books when I got home. I was also amazed by what Gabor Paller had managed to achieve with his Dedexer tool for Dalvic byte-code disassembly. The “park-bench” and Yan Minagawa’s talk about the future of mobile were particularly entertaining for the group discussion that ensued, and I’d never heard of the c-base space station that had crashed into the heart of Berlin – it sounds an awesome place. Finally, Diego Torres Milano’s talk on test driven development on Android was something I was especially pleased to see on the programme. I advocate and use TDD in my day job (and at the code dojo) – so it was great to learn how to do it on Android.
As always, some of the best stuff was in the chatting over pizza / drinks and in the intervals. Lots of cool pointers and advice as well as a sneak preview of Motorola’s “Droid”.
Great job guys and I’m looking forward to the next one…
(If you can’t wait there is always the Londroid group whose next meeting is on 21st January – I’ll be there).
On Tuesday and Wednesday of this week I attended 
