Battling the Platforms

It's autumn, rainy and windy so paragliding is well, over. Inspired by some recent encounters with interesting startups and a very nice presentation on experiences with the new Google App Engine, I decided to do a tiny project. And as all of a sudden lots of friends and acquaintances seem to have discovered Facebook, I decided to make a app for that.
With so many people on Facebook, I've noticed that it is a good source of interesting events: whatever one thinks of the rest of Facebook, I personally really enjoy to check out the events that my friends are interested in (the new Zipiko service is centered around this use case). Very frequently I want to attend the event too. But it just so happens that I use Google to host my master calendar and from there I sync to my laptop. Facebook offers "export" of events as iCal files but that's fairly primitive and above all cumbersome. So here I had a good candidate little project: make a Facebook app that reads future events that I've signed up for and add them to my Google Calendar. And host this app at Google App Engine (GAE). Although the functionality would be minimal the project would use both the Facebook and Google data APIs and libraries and use the Google App Engine web app development and hosting platform.

Doing a simple GAE app turned out to be fairly straightforward. A GAE app must be written in Python and that was initially my tallest hurdle; I'd never done anything in Python before. But I've years of Smalltalk experience and Python supposedly would be somewhat similar; and in any case not very hard. My Linux/Fedora laptop already had Python, all I needed was to add the PyDev plugin to my Eclipse install. Then I downloaded the GAE development kit, essentially a couple of Python libraries and a little, pure python, server that implements the same restrictions as GAE. Zipipop's Stefano's recent presentation on GAE discusses those restrictions at length; for this tiny project they were no issue at all. I found it easy enough to run the development server from within Eclipse, but I could not get debugging to work in- or outside Eclipse. Seems to be some issue with the Python version I'm using but GAE wouldn't work with a newer version. Anyway the project was going to be tiny so I figured I could do without debugging. In Smalltalk development it was very common to write most code in the debugger so I was curious to try that out with Python.

So once I had the obligatory "hello world" GAE app working on my laptop it was time to turn it into a Facebook app that could get my (Facebook) events. Just creating a page that would list the events was very easy with the Facebook Python library. The most important aspect was to build the app such that all requests could (and should) come from Facebook. In other words the browser/user should all the time be directed to Facebook which will then request the app to generate content for the so called "canvas".
In the app a few lines of library code are used to parse the HTTP body that Facebook POSTs. If the user is logged in (to Facebook) this contains a bunch of frequently needed user information. In my tiny project all we really needed was the Facebook user id, which was used in a single Facebook API call to fetch the future events (for that user). Facebook also offers FBML, a bunch of markup elements that make it easy to offer content in the Facebook style.

At this point I had Facebook events in the GAE memcache. So next was to insert those into the Google Calendar. Most of Google's services offer interfaces that all use the same unified approach: Google Data APIs. Google offers a Python library that nicely encapsulates most of the boring work. However it is important to more or less know what has to happen at the HTTP level, as things have to happen in the right order.
What needs to happen is essentially an OAuth dance. To get some real work done the app needs to present a (use) token and to get that token it has to present an authorization token. It gets this (one-time) authorization token by redirecting the user to the Google Account service with some parameters; the essential ones are a URI that indicates the scope of the actions that the app wants to do (i.e. "calendar") and a URL to where Google should redirect the user back. As explained above in my case this URL should actually point to Facebook, with my app as a URL path component. Upon succes Google will add the one-time authorization token as a parameter to that URL. Luckily Facebook puts this parameter in its request to the application. So in my little app I would catch the token and then upgrade it with a call to Google to a "use" token. That token is supposedly somewhat permanent so I decided to save it in a little User object. The GAE offers a object database functionality that deals with distribution completely behind the prying eyes of the developer , a very nice idea. Of course it didn't quite work right away...

This as the GAE documentation on the use of the Google Data Services states that:

Note: With Google App Engine, you must use the URLFetch API to request external URLs. In our Google Data Python client library, gdata.service does not use the URLFetch API by default. We have to tell the service object to use URLFetch by calling gdata.alt.appengine.run_on_appengine on the service object, like this: gdata.alt.appengine.run_on_appengine(self.client)

What is not obvious at all is that the data service client is then also tweaked to use a different token store, that itself tries to store tokens in the GAE database. To do that it needs to know about the Google user, but my app never authenticated the user as a Google user. It is after all a Facebook app, and I could key users on their Facebook user id. Needless to say this took me quite a few frustrating hours to figure out. Once I understood what was going on it was simple to modify that Python GAE library for the Google Data services a bit so that I could pass in a flag to indicate that I didn't want the normal client token store to be changed for the one "optimized" for GAE. In my app I then added the code to store and set tokens as I'd planned.

Now upgrading to a use token worked and with such token it is a simple Google Data services library call to insert an event, but there were some caveats. Most disturbingly was that when I ran the app on the GAE development server I always got a "Not Implemented" error. I assumed that this was because I'd done something wrong but when I after quite a while decided to try the app on the actual GAE infra this part immediately worked. The other caveat was that the Facebook API returns events with timing information in Pacific time. Sigh. After another hour or so I'd figured out how to use the Python libraries to move those back into UTC. I'm still not sure if I got this right for Daylight Saving Time situations.

A day later I suddenly noticed that it did no longer work, and quickly found out that this was because my code would sometimes get a trailing slash in the value of a URL request parameter. Easy to strip out but it hadn't happened before. I think this happened when Google released an upgrade of the GAE kit, but I'm not not completely sure that this problem co-occurred with that upgrade.

All in all it was an interesting exercise indeed! It would be tempting to compare the various platforms in detail but I've only scratched the surface. A couple of remarks can be made though. The Google App Engine seems a very useful hosting platform for not-too-complex websites. It forces one to structure the app in such a way that it will scale, but this is done in a positive, instructive, manner.
The Facebook APIs and the Google Data APIs are more or less comparable; they offer various identity-based services. Facebook can provides the profile of the user, the friend list, the events, and a couple of other things, e.g. a facility to send notifications and SMS messages. Google provides access to the calendar, documents, and contacts of the user. Unfortunately the Facebook and Google services have different interfaces although they are quite similar, simple HTTP request with JSON or XML for service response content. Overall the provided libraries look good and are effective in hiding the complexities very much. The disadvantage seems to be that the APIs are not specified with as much rigor as I would like.

Oh, and if you want to copy Facebook events into your Google Calendar you can try it at http://apps.facebook.com/toogcal. It might work, but don't count on it. Looks like tomorrow it will be flyable after all...