This morning I noticed a tweet from Cory Doctrow about a vote in the European Parliament concerning ACTA, the Anti-Counterfeiting Trade Agreement. As Wikipedia will tell you, it's a "proposed plurilateral trade agreement for establishing international standards on intellectual property rights enforcement throughout participating countries". Unfortunately, it's rather draconian, negotiated in secret and has an impact on our digital civil liberties.

Given that I was an a hurry with only minutes to spare I quickly followed this advice (given by Glen Moody who wrote the excellent Rebel Code). I scanned his example text and sent it off:

"I believe that there is a plenary vote today: I am writing to ask you to take that opportunity to sign the Written Declaration 12 on ACTA, which I feel is an important document that deserves your support for the sake of the European electorate and European democracy."

(This document explains what Written Declaration 12 is.)

Within an hour I'd had replies from three out of five (see Edit 1, Edit 2, Edit 3 and Edit 4 below) of my MEPs (or an official within their office): Glenis Willmott (from an official), Bill Newton Dunn (apparently from himself) and Emma McClarkin (apparently from herself). Nice one!

Whilst Glenis's official explained she will respond soon and Emma assured me that "the content of [my] email has been noted" I got the following interesting response from Bill:

"The plenary vote (which has just been completed) is not connected with Written Declarations which "lie on the table for many weeks" hoping to collect signatures. Could you explain, please, why the sake of European Democracy this needs our signatures? You say you "feel" but could you give me the arguments, please ?"

OK… well, as you asked Bill, here's how I responded (I've corrected a couple of typos):

"Dear Bill,

Many thanks for your prompt reply.

A bit of context: I am a former professional musician who now works as an independent software developer / consultant. As a result, the issue of intellectual property in the digital world is of interest to me on several levels (and I acknowledge that ACTA is broader in scope than just the digital world).

First of all, looking at the quick message I sent you makes me shudder as "feel" is definitely not the most appropriate term to use to bring your attention to the issue – let alone persuade you to act. To be honest, I was reacting very quickly to a "tweet" sent out by the well know author and activist Cory Doctrow (see: https://twitter.com/doctorow/status/11026659037). As you probably realise, proof reading and putting forth strong arguments do not mix well with a deadline only noticed with a minute to spare. ;-) Ergo, you'd be correct in thinking this was a compulsive act on my part.

In any case, as you asked, here are my fleshed out (yet pithy) arguments for why ACTA sucks harder than the proverbial black hole:

1) The secrecy surrounding the content of the treaty is wrong
2) It has the potential to infringe upon civil liberties
3) It's un-enforcable in any case

For each point in more detail:

1. Secrecy

Regarding my "for the sake of European Democracy"... Yes, this sounds bombastic but as the negotiations for this treaty are held in secret and I can only read the proposed wording of the treaty via leaked documents (see here for example: http://en.swpat.org/wiki/ACTA-6437-10.pdf_as_text) I fear that the open democratic process is being subverted. How, for instance, can I and many other like me whose work is directly affected by this treaty "officially" make our voices heard when its content is shrouded in secrecy..? (Granted, I'm writing to you.)

Whilst I understand the need for secrecy in matters concerning the military or security – surrounding ACTA with such secrecy is hard to comprehend especially given the impact it'll have on many of the EU's citizens. Unfortunately, it gives the impression that some sort of nefarious hood-winking is afoot… Surely, in an open democratic society citizens should be able to see what their elected officials and government institutions are up to on their behalf..? If this freedom of information is denied to the voter how are they to independently decide how to hold such elected officials to account come polling day? Hence, for the "sake of European democracy" I urge you to do everything you can to make ACTA open to public scrutiny.

2. Civil Liberties

Apparently (see my point about secrecy), the act will allow trademark and copyright holders to force Internet Service Providers (ISPs) to provide information about suspected copyright infringers without a warrant (i.e. due process). The problem here is when the words "suspected" and "without a warrant" are found in the same sentence. To my mind this makes it legal for an invasion of privacy. Hence my concerns for the European electorate (yes, another bombastic remark).

Border checks… fine for heroin, human trafficking and so on… but for mp3 files..? My gripe here concerns a (lack of a) proportional response to a problem (there are far more serious things our border officials should be checking on) and how such checks makes it legal for yet another invasion of privacy.

3. Enforceability

Put simply, no matter what ACTA says – it is relatively easy to circumvent DRM and distribution limitations. Furthermore, because of the nature of the digital medium, once a means to circumvent controls and distribute content is found it spreads quickly. While this is not my area of expertise, I know several people who most definitely are experts in this field and their reaction to some of the proposals ranges from disbelief to knowing amusement.

Concerning the problem of invasions of civil liberties – of course private citizens are going to push against this – see http://www.torproject.org/ for a good example of a mature and well known solution. My fear is that authorities will begin to contemplate control of such software that has quite legitimate uses.

I hope this answers your request for my arguments.

To be honest, this strikes me as a a classic case of political lobbying trumping an open democratic process. The film and recording industry who are promoting the adoption of ACTA have a business model that can't survive the Internet in the same way scribes couldn't survive the arrival of the printing press. Let's not prolong their demise with despotic legislation. Composers will still compose, musicians will still perform, films will still be shot, dancers will still dance, painters will still paint, writers will write and we will still think their work amazing. Please remember that our "culture" does not depend on an "industry" for it to flourish but it does depend on the freedom to share our artistic heritage. Try to imagine what Beethoven would have done if Haydn had trademarked the "String Quartet".

Finally, I hope you don't mind, but I'll share our correspondence with other friends of mine who live in your constituency (mainly in Northamptonshire). As I'm sure you'd agree, it's important to encourage and make visible participation with our European representatives.

Well done for getting to the end of rather a long email…! ;-)

With best wishes,

Nicholas"

As I explained at the end of the email, this blog post is my means of sharing the correspondence and I'll post updates should anything result from this exchange.

Edit 1 The other two MEPs just got back to me. Both answered as themselves: Roger Helmer simply explained he was absent and Derek Clark responded as follows:

"Thank you for your message. Yes I was in Brussels yesterday but I did not sign this written declaration. I do not sign written declarations because they always ask the Commission to take some kind of action but the Commission is composed of unelected individuals and I do not believe in handing them yet more authority.

You are quite right to draw attention to ACTA but there are hundreds of issues like this and they have mostly been resolved in the past by Governments acting together in common interest. I do not acknowledge the authority of the European Union."

Edit 2 Bill has just got back to me. Here's what he has to say:

Thanks !

Fame at last

And thanks, too, for the detailed explanation – which I have printed out and will study, before deciding "to sign or not to sign".

I've replied as follows:

Many thanks for the response and taking time to study my email. I and many others would be interested in how you decide to act and (more importantly) why. To many of us who are passionate about such issues the movement of thought by which our elected representatives arrive at such decisions is opaque. To my mind, this doesn't encourage engagement, participation or accountability – essential attributes for a healthy democracy.

I look forward to hearing how you reach your decision.

Edit 3 Bill's response is to the point:

How do I decide?

At Westminster where there is no separation of powers, MPs do what their party whips tell them to do (or they are cast into outer darkness, out of favour).

In a modern parliament, where whips must be polite, we make up our own minds. It is as, famously, Burke wrote to his electors in Bristol 200 years ago, "I am your representative, sent to exercise my best judgment… I am not your delegate, taking instructions…."

So, I listen to as many sides as offer me advice, also consult my own Liberal colleagues, and then make up my own mind!

I especially like the reference to Burke.

My reply:

Once again, many thanks for the prompt response.

The Edmund Burke quote is a great encapsulation of representational democracy and I'm pleased to hear you explain that MEPs have the opportunity to make their own minds up rather than follow a party whip. This is all the more interesting as you are listed at votewatch.eu as the Lib-Dem whip since 2004.

That's the "how" answered – I'd be most grateful if/when you make up your mind on ACTA to let me know the "why". Think of it as an opportunity to show that you "...exercise [your] best judgement".

I'm not suggesting that you do not exercise good judgement – rather, I simply feel it important that politicians provide evidence of judgement so that the electorate see what they're up to. Of course, you may disagree in which case I quite understand if I don't get a response! :-)

In any case, top marks out of all the East Midland's MEPs for taking the time to respond thus far.

Edit 4 Bill has emailed me to explain that votewatch.eu are out of date: he is no longer the whip. He also explains:

"In any case, EP whips are limited to being very polite, and having no powers!"

My last post resulted in the following question from John Erickson.

"I've only just discovered FluidDB and the reoccurring question is, ‘how is FluidDB different/better than CouchDB??'"

I'll try to answer as best I can.

The usual caveat applies when using a loaded word like "better" – it depends on what you want/need. As I can't speak for anyone else I'll only do a compare and contrast.

Lets concentrate on the similarities first… both databases are flat collections of items ("documents" in CouchDB, "objects" in FluidDB) with a structure based upon the classic subject/predicate/value triple:

CouchDB: Documents have fields with values.
FluidDB: Objects are tagged (often with values)

Documents <=> Objects (both are identified by a UUID),
Fields <=> Tags (both are dynamically typed),
Values <=> Values

A simple difference in terminology.

(Actually, in FluidDB an object/tag pair does not need to have an associated value. Simply associating a tag with an object can provide enough information. For example, the object that represents the book Dune might have the "ntoll/has_read" tag associated with it and as I am the only person within FluidDB to have permission to associate this tag with objects you can infer from the name of the tag that I once read the referenced book. I'm getting ahead of myself here, but it's important to point out that a tag-value is optional in FluidDB.)

The other obvious similarity is that both DBs have a RESTful API (hence the potential for CouchApps and Flow – web-applications as data hosted within their own database).

Now for the differences in no particular order:

1. There is only one FluidDB floating "up there" in the cloud and it holds everyone's data. With CouchDB companies and individuals are responsible for hosting their own instance(s) and such instances are usually created for a specific purpose/application.
2. The predicate part of the triple is very different. Fields in CouchDB are simply names of values. In FluidDB tags can be organised with namespaces. You start with an empty root namespace named after your username and create tags and namespaces underneath this. Just to be clear here, you can't associate an object with a tag that isn't yet defined – and yes, tags and namespaces are also objects in FluidDB so you can meta-tag. ;-)
3. Querying data is very different. CouchDB's "Views" are pre-defined map/reduce based algorithms. FluidDB provides an uber-minimalist (yet still evolving) SQL-like declarative language.
4. Security and permissions in CouchDB are document centric and define who can read data and what validation steps a user needs to pass before they can write data to a document (CouchDB also has the concept of an admin account). The "model of control" (as Terry Jones [creator of FluidDB] calls it) is FluidDB's killer feature: permissions apply only to tags, namespaces and tag-values. All objects are public and can be tagged by anyone.

The implications of point four are not at first obvious but it is definitely this feature that sets FluidDB apart from CouchDB and every other database that I know of. Because of this I'll spend the rest of this post illustrating why the model of control is so important.

Because all objects can be tagged by anyone lots of interesting information/behaviour begins to emerge and become possible.

Remember the "Dune" example? Because only I have permission to associate the "ntoll/has_read" tag with an object you can be pretty certain only those objects with this tag have been read by me. I can also control who sees my tags, their values and even allow other trusted users the ability to add, edit, and delete namespaces, tags and tag-values (in a similar way to being able to set permissions on a filesystem). This is important because it allows people to collaborate: users who are food enthusiasts could all agree to use the "foodies/rating" and "foodies/review" tags on objects representing restaurants. Only those users enthusiastic/trusted enough would be allowed into the group with permission to associate such tags. Furthermore, as "insiders" they might also have created a tag called "foodies/discount" that is only visible to them and, when attached to an object representing a restaurant, explains how to get a discount.

Even though some objects have an optional "fluiddb/about" tag (holding a unique value and set by the object's creator to provide some guidance as to what the object is supposed to represent) the only way to find out what an object really represents is to look at what tags and values are associated with it. The tags and associated values cast an outline of a sort of "data-shadow" identifying the object's referent.

For example, the object with the fluiddb/about tag-value "book:DUNE" might have the following subset of tags/values associated with it:

{

fluiddb/about: "book:DUNE",

ntoll/has_read, 

tim_oreilly/has_read, 

amazon.com/type: "Paperback Book",

amazon.com/title: "Dune", 

amazon.com/author: "Frank Herbert", 

amazon.com/isbn: "ISBN123456789", 

amazon.com/price: "$9.99", 

amazon.com/genre: "Sci-Fi", 

amazon.com/cover: (BINARY DATA FOR A PNG FILE),

tom/comment: "I really like the Sandworms", 

dick/opinion: "Far fetched and obtuse", 

sally/rating: 5,

books/other_editions: ['UUID for object x', 'UUID for object y', 'UUID for object z'],

books/isbn: "ISBN 123456789",

books/title: "Dune",

books/author: ['UUID of object representing Frank Herbert']

books/publisher: "Chilton Books",

books/first_published: "1965"

...

} 

Now, consider the following:

• In the same way that #hashtags and @username conventions have emerged on Twitter so conventions such as "username/has_read" will emerge within FluidDB (for example). I use it because a well known publisher of tech books might use it, along with many other people. Put simply, the schema of FluidDB evolves.
• Organisations that can verify they own a domain name can claim it as a root namespace within FluidDB. As a result Amazon has tagged the book with useful information.
• Tag values do not have to be strings… they can be anything and the mime-type used when HTTP PUTting them into FluidDB is what is used when retrieving them. As a result, the amazon.com/cover value is an image/png file (presumably of the book's cover). Actually there are two types of tag value: primitive and opaque – the difference is explained here (put simply, you can search against primitive values).
• The tags from the "books" namespace provide lots of useful information, much of it the same as that provided by amazon.com.
• The books/other_editions tag-value is a list of UUIDs identifying other objects related to this one in a similar way to how foreign keys are used to reference related tables/rows in relational databases.
• Lots of different people, organisations and applications have added tags to the object.
• I'll never see some of the tags associated with this object because I don't have permission to.
• If Amazon.com decide to delete their tags from this object it is still useful and all the other data is unaffected.

Another side-effect of the FluidDB model of control is that completely unrelated applications will be able to share data. This is already happening: Terry has written an application called Tickery that has imported lots of information from Twitter under the "twitter.com" namespace in FluidDB. Because this data is open for everyone to read I can make use of it from within Flow and carry out exactly the same searches as Tickery does (e.g. "has twitter.com/friends/jack and has twitter.com/friends/ev").

Suddenly the potential for mashing up data becomes huge and very interesting – especially as anyone can add further data to the objects Tickery and Flow have tagged.

In conclusion, we're all familiar with social networks – FluidDB is simply a social database ‘in the cloud' with its model of control as the secret magic sauce.

Since the end of last year I've been spending quite a lot of my spare time experimenting with FluidDB as a platform for web-development. The result is a still incomplete application that I call Flow (the source-code is hosted on GitHub).

Flow is a simple "show and tell" project to demonstrate how to write small javascript based applications for interacting with FluidDB. The aim is for Flow to become an application for writing javascript based applications with FluidDB.

I'll let you pause a minute while you work out the circularity of that last sentence. ;-)

To make things clearer I've cobbled together a short screen-cast that attempts to demonstrate what I've been up to. (It's very rough and ready, but you'll definitely get the idea.)

As I mention in the screen-cast, Flow makes use of the Sammy javascript framework, a modified jsFluidDB script, jQuery and FluidDB
itself (for hosting and data storage).

It's VERY incomplete, hardly tested properly, extemporised (rather than designed) and needs a lot of TLC – just a proof of concept really. Nevertheless, as the video demonstrated, you can:

• Sign in/out
• View, create, edit and delete namespaces/tags
• Edit tag values for mime-types text/* in all browsers and the ability to do the same for image/* and application/pdf and application/vnd.ms-powerpoint values in Firefox version 3.0 and above (more on this later).
• Search for objects using the simple query language that comes with FluidDB.

To use it properly you'll need to register for a username and password here:

http://fluidinfo.com/accounts/new

Here are some of the development highlights…

FluidDB as a Server

Lets just remind ourselves of how FluidDB is organised from the point of view of data / schema:

1. Objects represent things in FluidDB.
2. Users define attributes to attach to objects – these are tags that can be organised with namespaces.
3. When a tag/attribute is associated with an object it can (optionally) hold a value.
4. The value of a tag/object association can be anything but you can (optionally) specify a mime-type for each value.

This is all we need to understand for now. (FluidDB's "killer feature" is actually its permissions system and the implications thereof. It is so important that I'll save that topic for its own blog post later. But one step at a time…)

So how does FluidDB act as a web-server..?

Simple really (and apologies if this is too simple an explanation – I don't like to assume too much technical expertise)...

Remember that FluidDB is RESTful in that its API is exposed through the HTTP protocol (y'know, that thing the World Wide Web is built out of). So, if I want to retrieve the value of a tag/object association I simply HTTP GET a URL – just like a web browser does for web-pages.

So how about I create an object to represent my web-application, add a bunch of namespaces and tags that map to the names of the files and directories needed for the application to work and then associate these tags with my application's object? Of course, I make sure the tag/object values are the contents of the files and that the mime-types are set properly (e.g. values for tags mapping to .html files have the mime-type "text/html").

This means that if a browser does an HTTP GET request to the path for a tag/object value within FluidDB and the mime-type for the value is set to "text/html" it'll display it as a web-page. The same goes for javascript (application/x-javascript), CSS (text/css), images (image/*) and anything else (application/pdf for example).

Hey presto… FluidDB is a universal web-server.

Sammy.js

So we have a server for our application, but FluidDB doesn't allow any server-side scripting. This means we need to put our application's logic somewhere else – the only other place being on the client's browser.

Happily, we can use a browser's Javascript capabilities for this and doing so is not so much a leap as other well-known web-applications already do this (GMail, Google Docs, Zoho and so on).

What is more, the guys in the CouchDB project have been thinking along similar lines for a while and Aaron Quint has developed a javascript framework for browser-side web applications called Sammy.

Sammy is a tiny javascript framework built on top of jQuery. Because of a clever "hack" it is able to "trap" GET and POST requests from a page and handle them with javascript functions associated with "routes". It's certainly packed full of features for such a small application and I've been hacking away with a maniacal grin on my face as I've been exploring Aaron's work.

If you've still not figured out why this is important, take a look at these two images Aaron has created (extracted from this presentation and used with permission) to show the difference between "regular" web-application development and that promoted by CouchDB and (now) FluidDB:

Sammy allows us to live in a Lisp like "code is data" world with no web-server / web-framework playing piggy-in-the-middle. The database is the server and the framework is Sammy-as-data running on the client.

HTTP PUT, Forms, Mozilla & Binary Data

In order to create or update a tag on an object the API requires a PUT request containing the appropriate value. Unfortunately, if the data we want to send is not text based we have a problem.

The usual way to upload binary data is to use an HTML input element with the "type" set to "file". Unfortunately, forms can only use the POST and GET HTTP verbs (this seems an amazingly bad decision – surely they should only support POST and PUT..?).

Rather than change the whole Internet I investigated how to get at the raw data from the request with Javascript and re-frame it as a PUT request to FluidDB. It turns out this is impossible except for users of Firefox.

Since version 3.0 (or 1.9 of the Gecko engine that Firefox uses) it has been possible to do this with the getAsBinary() function. In fact Mozilla have also provided the rather useful getAsDataURL() and getAsText() functions too. These are definitely non-standard and more information about them can be found on Mozilla's website.

You can see how I did it by looking at the "upload_to_mozilla" function in the tag_value_app.js file in the javascript directory/namespace.

Unfortunately, this means I'm stymied for the other browsers and I'm not sure of the best solution. Of course, extracting textual data from a form is easy on all browsers so this only affects binary data. Nevertheless, it's a problem that probably should be solved if such database-hosted applications become popular.

That's all (for now)...

I'll keep plugging away at Flow and will post updates on Twitter.

I also have to give a presentation in March at ThoughtWorks in London about FluidDB. I was thinking… it might be fun to create a "presentation" application with FluidDB and Sammy. Something along the line of DOM slides, AJAX-S or S5.

Who knows… it might actually be useful…

Oh, hang on… some guy called Aaron Quint has already done it with CouchDB.

;-)

Last night was the fifth London Python Code Dojo. At the end of the December dojo we agreed to try a different format: a well defined problem to be tackled concurrently by small teams who have a pre-defined amount of time to complete it in (about an hour). This would be followed by a show and tell / compare and contrast session.

We hadn't decided on a problem before the dojo so we chose to follow a suggestion made at an earlier dojo. A white-board was provided for attendees to write up suggestions during the pizza / beer part at the start of the evening.

This seemed to have worked well and a consensus was reached to write an "old skool" style text-adventure game over the course of the next few dojos.

Why..?

Well, a classic text-based adventure game contains lots of opportunity to practice and experience different aspects of computer-science / programming. I can think of the following problems off the top of my head:

• Build and navigate a game world (a directed graph?)
• Represent and facilitate in-game objects / puzzles (e.g. "kill Grue with sword")
• Build a parser for interpreting player's commands
• Represent, control and validate game state
• NPC Artificial Intelligence / NLP (NPC = Non-player characters, NLP = natural language processing)
• Client / Server / Multi-user support (a la MUD / MOO – perhaps using Twisted?)
• Make a UI to display images and other multi-media

Last night we attempted to build and navigate a game world. The fruits of our efforts can be found here on GitHub.

Happily, the team based format worked exceptionally well affording more people the opportunity to write code and solve the problem. Furthermore, Dave Kirby kicked off the evening with a lightning talk about using CRC cards in small-group development teams – rather appropriate given the context. Thanks Dave!

The most entertaining part of the evening (for me) was the "show and tell" conclusion. Each team was given the use of the projector and five minutes to demonstrate the game, show their code (and tests!) and talk through their design and methodology. It was fascinating (and funny) to see how differently the teams had tackled the problem (four out of five had working examples) – yet there were also many similarities. Highlights included a solution created entirely through TDD, fancy-pants autocomplete courtesy of the CMD class, a solution that separated game data from the game engine and had its own (proprietary) file format and a game with some very imaginative writing, pirate voice overs and a very large octopus.

Great fun!

People have posted comments and photos on Twitter and Bruce Durling and Andy Kilner have also blogged about the event.

The next dojo is on the 4th February. Details can be found here: http://ldnpydojo.eventwax.com/6th-london-python-code-dojo

On Tuesday and Wednesday of this week I attended droidcon, the Android developer's conference organised by the friendly folks at Skills Matter and Kevin and Carl from Novoda – the most enthusiastic mobile software development company on the planet.

Tuesday was a barcamp and a great opportunity to meet people and learn from others. I especially enjoyed Carl's talk on his work using RESTful APIs from Android (something I've been experiment with in conjunction with FluidDB). Other highlights of the day included learning about the Zii Egg and recently announced Trinity device, seeing the worlds longest nested if/else statement (don't ask – very diagonal) and some really great chats both technical and "blue sky" about the potential for mobile (and specifically Android) devices.

The second day was a more traditional presentation-based conference and we had some great talks. Highlights for me were a couple of presentations that delved into and demonstrated the NDK (native development kit), both of which were musical in nature (spot on for me) and had me dusting off my C/C++ books when I got home. I was also amazed by what Gabor Paller had managed to achieve with his Dedexer tool for Dalvic byte-code disassembly. The "park-bench" and Yan Minagawa's talk about the future of mobile were particularly entertaining for the group discussion that ensued, and I'd never heard of the c-base space station that had crashed into the heart of Berlin – it sounds an awesome place. Finally, Diego Torres Milano's talk on test driven development on Android was something I was especially pleased to see on the programme. I advocate and use TDD in my day job (and at the code dojo) – so it was great to learn how to do it on Android.

As always, some of the best stuff was in the chatting over pizza / drinks and in the intervals. Lots of cool pointers and advice as well as a sneak preview of Motorola's "Droid".

Great job guys and I'm looking forward to the next one…

(If you can't wait there is always the Londroid group whose next meeting is on 21st January – I'll be there).