I've recently got back from this year's Europython conference. It's a bit of a long post (there was so much good stuff), but here's my round-up of what I learned, found interesting, intriguing and cool.

FluidDB

FluidDB is a fascinating (if unintentionally secretive) project that I've known about since watching this video. The "talking head" is one of the founders, Terry Jones. I was so intrigued by what I saw that I emailed Terry a bunch of daft questions and he was kind enough to reply.

As a result, I was looking forward to two talks about FluidDB: Terry's own Introducing FluidDB and Esteve Fernandez's Twisted, AMQP and Thrift. The former being a high level "philosophical" view of whilst the latter being more concerned with some of the underlying (Python based) technology.

I have to admit that it is the "philosophical" aspect of this project that has me hooked. "A database with the heart of a wiki" is the tag line of the project, but, as far as I can tell, that only scratches the surface (Terry also describes it as a metadata engine for everyone and everything). If I understand Terry correctly…

• FluidDB is not a relational database but stores sets of tag / value pairs.
• Each set represents a "thing". All "things" are public in that anyone can add tag / value pairs to it.
• All tag / value pairs are protected by a strong yet simple permissions based system.
• Tags are also "things" that can themselves be tagged (higher order / meta tagging).
• Tags are organised in namespaces that are owned by an account (for a person, organisation or application [for example]).
• It is schema-less in that the tag / value pairs associated with a "thing" are not predefined.
• It is an "open" system in that any account can add data without having to ask permission.
• Retrieving data is easy through the use of a simple query language.
• It's been designed to scale.

Given such a back-of-a-postcard (and probably inaccurate) description – why is FluidDB so interesting? Two things immediately strike me:

1. Free to write, with finely grained contributions (by looking at the tag/namespace one can tell who has contributed what).
2. Evolution as a means of database development.

Free to Write – Anyone can add a tag / value pair to a "thing". However, you might not be able to nor want to see everything associated with a "thing": you might not have permission to view some of the tags from certain namespaces and you might only be interested in those from other namespaces.

Evolutionary development – Because of the "fluid" nature of FluidDB conventions and practices emerge due to evolutionary pressure in exactly the same way they do for our wider social conventions. This is important because, until now, database development (and thus, the way information is organised / presented) has been decided by the likes of me – a software developer – and there isn't any guarantee I'll do this in a way that is useful for you (either because of my lack of skill or because I want to retain / impose control and also because it's impossible to anticipate what people will want to do with things).

Perhaps an example might shed some light…

So a "thing" (Set) consists of tag / value pairs from various namespaces..?

Absolutely!

But what does it represent?

Whatever the tag / value pairs seem to imply.

Actually, there is already a helpful convention for working out what a thing represents: a special immutable tag called "about" whose value is unique and can only be set when the thing to which it refers is created. It allows you [and everyone else] to ask for an object about X where X is something helpfully unique like "NASDAQ:GOOG".

So, if a "thing" had a "nicholas/title" tag with the associated value "Seven Pillars of Wisdom", a "nicholas/author" tag with the associated value "T.E.Lawrence" and another tag called "nicholas/ISBN" with the value "0954641809" then you can be pretty sure that I am attempting to describe a book (the tags exist within my "nicholas" namespace). Furthermore, the special "about" tag might have been set to "ISBN:0954641809" when the "thing" was first created – indicating the thing is a book with a particular ISBN.

Contrast this with a regular database schema or API (such as that from Amazon.com) and you'll notice that they have already defined the concept of a "book" as represented in a "books" table with certain fields defined with certain types and perhaps a many-to-many relation to an "authors" table or other "helpful" stuff. The conventional system is imposing structure whereas FluidDB does not: you invent your own.

For example, the same set might also contain the following additional tag / value pairs along the lines of:

Tag	Value
wahida/location	"Wadi Rum"
wahida/long	29.5765
wahida/lat	35.419928
amazon.com/Product Description	"Seven Pillars of Wisdom is the monumental work that assured T.E. Lawrence's place in history as "Lawrence of Arabia." Not only a consummate military history, but also a colorful epic and a lyrical exploration of the mind of a great man, this is one of the indisputable classics of 20th century English literature. Line drawings throughout."
amazon.com/Average Customer Review	4.5
fred/rating	"10/10"
jill/score	"5 stars"
bill/genre	"Autobiography, War"

Wahida is obviously describing the rock formation that is also called the Seven Pillars of Wisdom and has provided the coordinates for the location.

Amazon.com and Bill have attached some product information and Fred and Jill have both indicated positive opinions.

How do we know the namespace Amazon.com is associated with the bookseller with that domain name? I'll let Terry answer that question (quoted from an email):

"Fluidinfo will only give namespaces that match domains to the actual domain owner, [so] you'll know that's an official amazon tag. That's part of allowing the evolution of reputation and trust – with a giant headstart seeing as we get to import the trust associated with internet domain names."

Interestingly, Fred and Jill use different tag-names and scoring systems to represent their opinions. Furthermore, they don't make it clear to what they refer that has the name "Seven Pillars of Wisdom" (although we'll probably assume they're referring to the book rather than the location).

By adding such information we have an example of "Free to write" and an evolving schema.

Assuming all these tag / value pairs are public, then I could start to cross reference information such as the rather obvious "best reviewed books by T.E.Lawrence" to the not-so-obvious "books associated with places in Arabia". I might also be buddies with Fred and would like to find out what he likes to read but find the publicity bumph from Amazon misleading so select to search using only Fred's tags.

Over a period of time Wahida might create a new thing about the rock formation, move her tags / values over and associate the original thing (the book by T.E.Lawrence) with the new one via a tag called "named after" that stores the unique ID of her new "thing". She does this because she's noticed that other people are making a distinction between a place and things associated with that place.

Also, Jill decides to use the tag name "rating" rather than "score" and to give her marks "out of ten" because that's just what more people do. Notice that the database "schema" changes as conventions become apparent over time. This is evolution at work as the fitness of the convention is determined by how useful it is to the people namespace authors care about.

Is this such a strange idea? Absolutely not, it's exactly how we get stuff done in our wider "social" world – for example, like the philosopher John Searle talks about when he refers to social reality.

I'm pretty sure I've missed something or not fully understood everything. But then, with no documentation or implementation to explore I'm doing nothing more than recollecting and guessing. Nevertheless, we were told we would get both these things "in a month". [Edit – Terry has been in touch and given me a sneak peek at the docs – unfortunately I haven't had chance to read them yet.]

Finally, I spoke to both Terry (just after his talk) and Esteve (in one of those really interesting corridor conversations) and their attitude reminds me of a quote from the introduction to T.E.Lawrence's book mentioned above:

"All men dream; but not equally. Those who dream by night in the dusty recesses of their minds wake in the day to find that it was vanity; but the dreamers of the day are dangerous men, for they may act out their dreams with open eyes, to make it possible."

Terry and Esteve are dangerous men!

Best of luck with your venture guys and I look forward to playing with it in the not-too-distant future.

Pythonic Music

I'm a classically trained musician so so I'm always interested to hear about tools for musical composition / development. There were two examples of this at the conference: RjDj presented by Chris McCormick and a display in the foyer for Ableton Live (who were looking for developers to join their team).

RjDj is an iPhone application that uses "scenes" (akin to musical "recipes" that act as proxies for the original composer) to take input (via the microphone or motion sensors) and mix it with other sounds provided by the composer to produce what I'd call an "auditory experience" that is always unique.

Chris's boss demonstrates the product in this video (at about 4 minutes in you get to see him use pens, staplers and his mug as the sources of input sounds into a scene):

All very cool.

What has this to do with Python..? Whilst the audio libraries are coded in C++ and the iPhone application is obviously in Obj-C all the glue code is in Python (including a Django based website).

I get the impression that Chris hates developing for the iPhone as we were treated to a very entertaining rant targeted at Apple (I can't wait to get this on Android!).

I also ended up chatting to the Ableton guys during lunch on Thursday. When I explained I had a musical background they gave me a quick demo, handed me a demo CD and told me to check out the site. Apparently, much of their software is written in Python with only the high-performance audio functionality being written in C/C++.

I've had a lot of fun playing with the demo version – I followed a couple of tutorials and then did my own thing. I also had a look around the web-site and their promo-video pretty much sums up what their product is capable of.

Un-conference / Corridor Chats

For me, it is often the corridor chats and conversations over lunch that are the most rewarding aspects of a conference. Europython 2009 was no exception: speech recognition and natural language processing, Zen Buddhism, FOSS in a corporate environment, Django / Pinax, music and software apprenticeship were just some of the topics covered.

Bruce Eckel gave a quick pre-keynote presentation on unconferences – conferences based around a theme and consisting of participant-led talks and presentations. Sounds like my kind of place and, happily, PyConUK this year will be organised along these principles (happening at the same location at EuroPython sometime in September). Can't wait!

Bletchley Park

I live about 12 miles north of Bletchley Park (in Towcester) and visit several times a year with my kids. As well as being important from a historical perspective (the centre of British code-breaking during WWII and home of the world's first programmable digital computer) it is also a "grand day out" with lots of things to see and do.

So I was especially pleased to hear the keynote by Sue Black and Simon Greenish about Bletchely Park. Unfortunately, not enough people know about this gem of a museum or its continuing financial woes. The attendees of EuroPython strike me as a sympathetic audience to pitch to and the resulting interest in the authentic Enigma machine (see photo above) and a trip to Bletchley organised on the conference mailing list demonstrated Sue and Simon are onto something.

Lets hope they continue to make progress with their fundraising efforts.

Turtle

As you might have noticed, my daughter and I have an interest in Logo, turtles and other such fun. I was especially pleased to attend the talk turtle.py – a Teaching Tool given by turtle.py's creator Gregor Lingl.

Put simply, Python now has a module that engages with kids like my daughter. Furthermore, she can play – as kids should – in an environment that encourages play as a means of learning. Finally, she can "graduate", should she choose to do so, into other modules and the wider Python language. Turtle.py is worth its weight in gold (yeah – I know source code doesn't weigh anything) simply because it facilitates the transition from childlike playfulness to the playful creativity that is programming.

I was also entertained by Gregor's examples: As he showed off ever increasingly complex software written with the turtle.py module I came to the realisation that everything in Gregor's world is a turtle. This appealed to my sense of humour – especially when he put up a slide with the text "A Website" and I mistakenly thought he'd written a simple web-server "out of turtles" (I was wrong – he was asking for help with the project website).

IronPython

Prior to my Python work I was a .NET guy so I was particularly looking forward to Michael Foord's Introduction to IronPython (it turns out Michael lives quite close to me: two Pythonistas in rural Northamptonshire..? There must be something in the water).

I have a little knowledge of IronPython from my days at Barclays Capital: we looked at including it in a tool I had built so our end customers (other developers in the bank) could customize various aspects of the software. In the end we had other priorities and I left the bank to learn Python.

Michael managed to pitch it just right to newbies like me with feet in both "camps". I was especially interested to hear how IronPython consolidates itself with non-Pythonic aspects of the CLR (method overloading for example) and integrates with the wider .NET framework. It certainly whetted my appetite and I'm now chomping my way through some of the chapters in his recently published IronPython in Action (a very good read – although I'm still trying to work out what the guy on the front cover is all about, there isn't a colophon like in O'Reilly books).

Thanks

I want to end with a big "thank you" to all those involved in the organisation and execution of the conference – I had a great time and I'm already looking forward to next year. ;-)