Custard Melt

Geonames are in

We've managed to migrate to the data from www.geonames.org. With a lot of places and a, let's say, less-than-optimal data model for geo data, it took some effort to import the data and migrate the current content. Thanks to ground work of Robin of Sonologic, and some more ploughing through data by Tim here, we now have a completely new set of place names.

]]> It also comes with some good lessons learned. My fear with the minimalistic approach of agile was grounded. Apart from the rules of "you ain't gonna need it" (YAGNI) and "don't repeat yourself" (DRY) we need to give more weight to DITWA: don't invent the wheel again. Looking at the database tables with Tim, I had the feeling "it's deja vu all over again"...

At the very least we will have to fix our data model to be completely compatible with Geonames. Saves a lot of hassle for updating. But better still is to make this the first real test for interfacing with other web services. Geonames already found out how to point people searching for "Moscow" to Russia rather than Moscow, Alabama in the US. And just become better in caching results rather than building the same functionality.

So... on with the show, Simon of 8Media has been working on several issues over the last weeks, and we'll be working with them for another few weeks to clean up more bugs and usability issues. Then it's about time to get that to the CoolThePlanet site. With the RSS feed from our subversion server in my Miranda IM contact list, it's nice to see the message icon blinking every day for new updates coming in.

Moving to Geonames

We've been struggling with the geo data for a while, after compiling a database from various sources ourselves. A lot of work, very doable, yet usually beaten by other work on people's priority lists here. Over the last months, www.geonames.org seems to pick up more and more steam, and so we decided to move over and team up with the community maintaining that data. Tomorrow, we'll start migrating the current database content, and have a first go at staying up-to-date with the geonames.org changes.

There is some python code available to interface with the geonames web services, so that fuels my hopes we can also manage to link up in a more "web 2.0" way with the data services over there. It's about time we have some more mashing up than just our Google maps. If anyone is already working on a Django interface for this, I'd love to hear from you!

CoolThePlanet silent launch, improvements and usability

It's been silent on the Melt blog, partly because I was unable to work for a few days, partly because we did a silent launch of www.cooltheplanet.net, and invited first people to test it out. In the meantime, we also worked on Speakup Middle East, our sister incarnation of the "Custard Melt" software. And we did a first round of improvements and fixes based on user feedback. We have a special group for that, by the way.

]]> This week, we have Duncan, Jeremy, and Felix from ThoughtWorks over at the Greenpeace office, and Wednesday, Simon from Eight Media joined as well: discussing merging back the little bit of forking that we already introduced, getting the tests to run and be less brittle, and making the skinning more separated and easier. The next weeks, we'll do some feature improvements on forums for SpeakUp, while CoolThePlanet will be taken to the streets: Al Gore's Inconvenient Truth will soon be released globally, and many groups (including Greenpeace groups) want to take the opportunity to point out what you can do.

Tim and Alex here at Greenpeace have been spending a lot of time with Jeremy on usability and flow of navigation. Alex spent a week with our Mediterranean staff on SpeakUp, so I've been participating as much as possible to bring in the first lessons from CoolThePlanet as well, and plot a common path forward on usability.

Better sign-up process, new homepage, better navigation, less trying to show off with our google maps, rss, easier linking, there's a lot in the pipeline again. But first finish writing some of the "static texts" now...

Go-live date, second deployment and other stuff

Well it's been a longer than expected road but we're nearly there. Our staging instance is running something that is 99.9% of a production branch. We would have gone live on Friday but staffing shortages mean it's going to be Tuesday now...

In other news we're working on a second instance of the platform, this one to support work Greenpeace will be doing in the middle east in the wake of the Lebanon crisis. You can read more about that project at it's holding site at www.speakup-middleeast.org . Hopefully that site will be up and running by the end of this week.

We've also been working hard to import our geo-data into the system. The GNS data we've been working with varies widely in quality. Some countries (eg Australia) can be pulled seamlessly out of the database, others (eg. Israel) suffer from duplicate or missing administration districts which has necessitated some manual fixes. The GNS data provides about ten times more data than the initial Geo-Rosetta data we deployed, so it's worth the hassle to get it in there.

Still, nearly there...

Second skinning

Yes, this week we are moving! Second-skinning the Melt site, and you can see it happen. After quite a few discussions and deliberations about name, logo, colours, what not, we're now at my preferred stage: doing it. Right now, I am at the office of Eight Media in Arnhem, where Simon is working live at the Melt staging site to implement the design. You will notice the name under which this all will go live: CoolThePlanet.net. Lots of little design decisions and checking it out with BrowserCam on different platforms, while hacking away at the Django templates, hopefully taking the opportunity to internationalise the static interface texts in there, and be ready for localisation in other languages. But first, counting down the days now to the opening of CoolThePlanet.net!

Getting up to speed for developers

As we're moving along, I'm starting to collect stuff that can help people get started with Django. An interesting list of links on the blixtra blog: Top 30 Django Tutorials and Articles, complementing the overview of documentation on the Django site.

In the meantime, our colleagues at hosting have provided us a virtual machine to configure as the host for our platform. We are thinking about making such a virtual machine available to developers as well. It would make live so much easier: download the image, run it, and you're ready to help in developing our platform. No other downloads or setups needed, eveything including the automated testing ready for you to use.

Also, the data sets for our geo-location data are being merged. We have some 108,000 places in there, and about to add another 150,000 or so for the US, with a third collection still waiting. Getting everything together (longitude, lattitude, duplicate names, missing links to country or region, different spellings or transcriptions, there is some work involved before we get things on a map). Another interesting week ahead.

Back online and moving ahead

Last Friday we had a problem with our development server, and so our Melt instances were unavailable for a while. Yesterday, a disk change in that server caused some more downtime, but things are back up again.

In the meantime, it has been a bit quiet here on the blog, while work was progressing. The naming and design are moving forward, and we should have final versions and be implementing the design in our templates soon. And... we now have the Melt project space at Sourceforge (thanks for your help, Misha!), so our software will become available through the new Sourceforge Subversion system.

"Show and tell" this Thursday

This Thursday, the basement at our Greenpeace headquarters will be filled with Ruby and Django folks, coming for a "show and tell" meeting to talk about their projects. From 14:30 until 16:30 there will be six short talks with Q&A, including Ximon explaining how we work with Ruby here.

In the meantime, we're coming close to the "show and tell" phase of our platform. The last "blocking issues" for a first live release are being solved this week, and we're making progress with a design overhaul. OF course, that leaves us with only a million or so new ideas to develop on the platform, so we're also moving the open source side of the platform to a new platform to open it up for more people to contribute.

Keep an eye on this blog, or get in touch with me if you can help us! (rolf.kleef (at) int.greenpeace.org)

NetSquared Online Sessions

I am in San Jose now, just a few hours away from the opening reception of NetSquared, probably the biggest "Web 2.0" conference I will be attending for a while, with some 350 participants expected. Many many interesting talks and sessions proposed, including a parallel online event, with two important items.

Tuesday 30th May, 11AM PST (which is 8PM Western Europe, 7PM UK, 6PM UTC) I will have an hour session on our Custard Melt project, and especially on what our ambitions and expectations are. I hope that many people will join to explore how we can make the online platform help in galvanising offline action.

At 4PM PST, 1AM Western Europe, midnight UK, 11PM UTC, Micki Krimmel of Participant Productions will be online for a session on "Media that Mobilizes: An Inconvenient Truth, ClimateCrisis and more tales from Participate.net". Of course, as the producers behind Al Gore's movie about climate change, this is of particular interest to us.

World Cup on your Palm

A bit off-topic, but for those of you (like me) with a Palm-OS-based organiser: if you want to have the schedule of all World Cup matches available, keep track of scores, add the matches to your agenda, and even download them wirelessly, you might want to look at SK7Software's "World Cup 2006". They give the software away for free, but ask you to take action against climate change, refering to our colleagues at WWF in the UK.

New spreadsheet

Getting back underway has seen my faithful project management spreadsheet undergo something of an upgrade. First Tim (our new lead at ThoughtWorks) added a number of tabs for tracking individual iterations and graphing our velocity - which will be interesting to watch as the new guys get up to speed.

Secondly I split out the non-functional stories onto a separate tab, since I've never felt quite comfortable having things like 'Must use standards compliant xHTML' sit next to 'Should be able to order search results alphabetically', it just didn't seem like we were measuring the same things.

Finally I've added the bugs we've found as stories in cases where they can't be listed directly against an existing story. I'm not sure how that's going to pan out but we'll see as we go...

You can see the new spreadsheet at http://svn.greenpeace.org/repositories/custard/doc/Custard%20Master%20Story%20List.xls?rev=592

Back underway...

Following an extended hiatus the project is back in full time development. Once again the work is being done by ThoughtWorks. In addition we've engaged the services of Amsterdam based Total Identity to provide identity, look and feel and similar things for the launch. On the back end work has started on getting a production ready server set-up in place for the application.

Greenpeace at Google

About eleven minutes and thirty seconds into his talk on Django to Google Django creator Jason Kaplan Moss talks about Melt. So it's nice to get noticed.

As a more general update we're planning to get back to work on Melt this month, with a target of getting the first public site up next month. During our recent downtime we've been gratified to see Google adding more data to their Google maps, so that the system makes much more sense now when you're looking at results in Europe.

Housekeeping

It seems that this blog has started to attract a bit of comment spam and a lot of trackback spam. I've deleted it all (apologies if any genuine comments have gone by accident), and added authentication requests onto the commenting features using TypeKey. I think I'm just going to have to turn off the trackback feature until I have more time to look into this.

If anyone can recommend a good moveable type compatible service for captcha tests that would be great - seems preferable to asking people to get a username / password.

Renaming Climate Change

Seth Godin points out that what we should be talking about isn't climate change, it's 'Atmosphere Cancer' or something equally serious sounding.

While I don't want to sound flippant about seriousness of this he may have a point. After all, the Great Bear Rain Forest used to be called the North East Timber Supply. Changing it's name was a key part of saving it.