I Won the Podio API Release Hackathon Competition

Introduction

On May 14th, I competed at a Hackfest hosted by Podio. Podio is a customizable social networking application delivered as a service (aaS). After 8 hours of coding, I placed first in the competition and won a beautiful Apple Cinema Display. In this post I will cover my hack, how it was done, and lessons learned.

Victori pretium it

Background

Let me first introduce the various companies involved and technologies provided or implemented in the solution. I think of these as the ingredients in the recipe that allowed me to code my offering.

Podio

In case you haven’t noticed, people are spending phenomenally large amounts of time on social networks. These applications have their own vocabulary of interactions and provide certain defined workflows that, thanks to their popularity, are teaching the population certain workflows for successful collaboration. As younger employees enter the workforce, their models of collaboration will be informed by these social networks whose existence and interaction vocabulary are presumed to be ubiquitous. Accordingly, it would be wise for businesses to adjust their workflows to look something like these social sites.

The traditional solution to this has been to acquire servers, place them in a data center, and build your own social networking clone which, invariably, would be perceived as a dismal dog in comparison to the glittery product like Facebook, Linked In, et al. distilled in the market crucible. In short, building a social platform is probably not your core competency.

Enter Podio who offers a software solution that behaves “like Facebook” and which can be used as a service.

Traditionally, when using an aaS solution, one loses the ability to create highly specfic applications (“Podio’s not going to build an app to show what’s in our cafeteria today”). Or one might expect that to get a specific app one would have to pay Podio a handsome consulting engagement fee. Podio has seen a way around these drawbacks of using an aaS service by providing an in-application, friendly, web-based, drag-and-drop application builder. This is the first key element in my solution.

While a well-conceived application builder is great, certain behaviors are unique to certain businesses (e.g. “SMS the sales manager if no one takes ownership of the customer complaint within 24 hours.”) and don’t have an obvious integration point in a web interface. To remedy this concern, Podio provides a rich API (Application Program Interface) that’s a pleasure to use. It allows 3rd parties to extend and share functionality between the Podio core and their own custom applications. It was the public release of this API on Saturday that occasioned the Hackfest. It was also the second key element in my solution.

On the topic of the API, let me add that many technical entities develop an API as an afterthought and the reliability and consideration (or lack thereof) shows. Podio bravely (!) decided to re-implement their web interface as a client of their own API. This is a brave move, but by testing whether they themselves could work with the API they were providing, they wound up creating a better, more-intuitive app.

Incidentally Podio is not purely a virtual experience, they have the Podio Store in San Francisco’s SoMa districit where you can go and talk to their staff about what you could do with Podio. They’ve made the “app store” software conceit physical. It’s revolutionarily transgressive in a good way :)

Twilio

Twilio is a service vendor who provides an exceedingly simple-to-use API for interacting with phones: setting up phone menus (“Press 1 for …, Press 2 for…”), making phone calls, bridging calls, setting up conference calls, setting up answer menus, etc. is a breeze. If you need to do something that touches a phone, Twilio’s rich, well-documented API will make it easy.

I had gotten a Twilio developer key and $30.00 of free minutes that I hadn’t touched since it was given to me at the Social Apps workshop held at Heroku in January 2010 (ulp). Twilio’s API was the third component in my solution.

Heroku

Heroku makes it incredibly easy to stand up and host web applications. I have only ever used it to deploy Rails and Sinatra applications. Heroku makes a very complex process managable, but it is not without some complexity. Heroku’s service was the fourth and final vended technology involved in my solution.

Previous Experience

It is fair to say that I probably spent the most of my time on the hack day trying to figure out the right way to develop with Heroku. However, if you read through this post and reference the implementation process, while not completely effortless, you will capture the value proposition posed by Heroku’s use.

Previously I had used Heroku to host my Verba Latina in Viis Ferrorum (Latin on Rails) application that I used in a failed bid to get into grad school (Cue violins and tears). As such I was familiar with their streamlined, git-based deployment design as well as with the concept that you must configure your app to have a configuration manifest so that Heroku builds the application the right way.

Last January Heroku and Twilio hosted an introduction to using Twilio and Heroku to create social, phone-based applications. Here a pattern emerged whose introduction was very important to my Podio solution:

1. Acquire a Heroku instance
2. Run a web framework (the lighter the better, for debugging purposes)
3. Use your instance to bridge communications between API’s
4. Violà, mashup

However at the Social Apps workshop, I didn’t really do this, instead I learned about some other Ruby tools described below. While the pattern would have been good to have had some experience with, the following tools were also essential to my success at Podio, so it was good that I spent time learning a bit about them.

Ruby Tooling

Sinatra

Sinatra is a very light language for describing a web server. You say: hey, when someone points to you with / at the end, send back some text generated by executing some code. It’s incredibly simple, incredibly clear and I had done a “hello Heroku” on it as part of the Social Apps get-together.

Ruby

Sinatra is written in Ruby, so, I’m glad I know Ruby.

RVM

RVM is a way to have multiple Ruby interpreters installed on a machine at the same time such that their various libraries and dependencies don’t interfere with one another. It also has a wonderful tool called “gemsets” which allow you to store all your Ruby gems (libraries for specific tasks) in a single sandbox so that you know what you have when it comes time to deploy.

git

Git is for revision control. More than once I screwed up a file and needed to get back to a working state quickly, specifically because the competition was timed. The ability to not have to do dozens of ‘undo’ operations was critical.

vim

A fast editor for editing text. ‘Nuff said.

Ingredient List

Having shown the background above, I had the following ingredients when I showed up:

1. Podio:
   1. In-application, web-based, drag-and-drop application builder
   2. API
2. Twilio
3. API
4. Free developer minutes from Social Apps workshop
5. Heroku
6. Sinatra
7. Other development tools

As of Saturday morning, I would rate my proficiency with Podio, Twilio, and Heroku, and Sinatra as being very low. It is a tribute to the designers of these systems that I was able to stitch them all together in just a few hours.

The Hack

Kick-Off

First things first I arrived at Podio’s beautiful storefront on 6th street, SoMa, SF, CA, but 2 blocks from my house, at 8am and was treated to some great brain-food for breakfast. The interior is beautuful with Scandinavian sensibility and with several ukelele’s decorating the wall. The loft-style space is modern and tasteful and a great place to visit.

The ukelele wall

The morning started off with Phil Chambers (CTO, @filchambers) and Christian Holm (Developer, @hlmrn) showing off what the Podio drag and drop tool and API is capable of.

Christian demonstrates

Now here’s a bit of a sheepish confession I have to make: I hadn’t really even considered there was a competition. I thought it was going to be a meet and greet, code some, and get some tasty food and / or beverages…but then Phil mentioned that there were prizes for the best use of the API! To sweeten the deal with the spectre of avarice, Podio had brought in some great prizes: Dr. Dre headphones, an iPad2, and a beautiful Apple Cinema Display. I figured going home with headphones wouldn’t be bad for a day’s work. I half-flirted with maybe even getting the iPad but didn’t put much store by it. Besides, I had work to do.

Ideation

After the “Go!” bell was rung, I thought about something that had been on my mind. They day before I’d had a frustrating visit with my HOA. I found myself wishing that there were some way for us to keep better track of our “shared” maintenance tickets (handled via email). The first thing I did was use Podio’s drag-and-drop application builder to build a basic ticket tracker. While most tickets are private and bonded to a single resident, there should be “shared / public” tickets for shared issues: water stains in a hallway, Insanity Wolf in the garage, etc.

A fearsome menace! Insanity wolf does not read your Tumblr blog

I called this app “Is It Done Yet?” – a common-enough thought among most urban flat-dwellers when considering the state of their work orders with their property management companies.

By using the Podio app builder …

I was able to create “IIDY?” tickets…

This took me all of about 15 minutes to put together. It was about this time that Jon Froda, one of the founders, came by to talk to me about how things were going. I explained the work-order tracking idea and he suggested that maybe I could add an alerting system for “Emergency” requests and that I could have that page out. Having been a sysadmin, I knew about pager notifications and while I thought it might be a bit too simple to be a real competitor, I knew that it was implementable in a few hours…provided I use Heroku and Twilio. I was a bit daunted since I was not too familiar with these peer components and I certainly had no experience in tying them to the Podio API.

…But Jon’s idea seemed audacious to win a prize but was slim enough to do in a few hours. I went with it.

I thought the idea over and sketched up a basic schematic for the design with the help of Christian. I worked from right to left.

Podio                    Heroku-hosted Custom App               Twilio
 Web Hook On Create -->    Receives event                  |-->API sends SMS
                           Evaluates emergency Status      |
                           If emergency, paged via Twilio -|

Set Up the Development Space

$ mkdir podio
$ cd podio
$ git init
$ echo '\*.sw\*' >> .gitignore
$ touch Gemfile Gemfile.lock
$ echo 'rvm use ruby-1.9.2-p180@podio' >> .rvmrc
$ cd ~
$ rvm use ruby-1.9.2-p180
$ rvm gemset create podio
$ cd -

info: Using ruby 1.9.2 p180 with gemset podio
$

As I mentioned earlier, knowing some core Ruby / Unix tools made getting started and working efficiently an edge for a team of one. These command set creates a directory, a primitive git ignore file, and creates a Gemfile that will be used for defining the apps’ gem dependenices via bundler as required by Heroku. It also specifies an RVM gemset which will sandbox all the dependencies.

Send an SMS via Twilio from the CLI

Twilio provides an Ruby wrapper for dealing with their API. The first thing I did was get the twiliolib gem. I then found the Twilio SMS example that, although written in PHP, was sufficient to give me a place to start.

I implemented this code like so:

#!/usr/bin/env ruby

require 'twiliolib'
require 'pp'

# Twilio REST API version
API_VERSION = '2010-04-01'

# Twilio AccountSid and AuthToken
ACCOUNT_SID = 'MY-SID'
ACCOUNT_TOKEN = 'MY-AUTH-TOKEN'

# Outgoing Caller ID previously validated with Twilio
CALLER_ID = 'MY-VALIDATED CALLER ID';

# Create a Twilio REST account object using your Twilio account ID and token
account = Twilio::RestAccount.new(ACCOUNT_SID, ACCOUNT_TOKEN)

d = { 'From' => CALLER_ID, 'To' => 'MY-CELL-PHONE', 'Body' => ARGV.empty? ?
'An EMERGENCY severity task has been opened!  Please check PODIO task list.'
: ARGV[0] }

resp =
account.request("/#{API_VERSION}/Accounts/#{ACCOUNT_SID}/SMS/Messages",
'POST', d) resp.error! unless resp.kind\_of? Net::HTTPSuccess

puts "code: %s\nbody: %s" % [resp.code, resp.body]

What could be easier!? Well, I got killed by a bug here that took a lot of debugging. I had put in the wrong CALLER_ID value. Make sure you cut and paste the validated phone number value into your code. You can find the number on your Twilio user page.

Tip: In the last puts of the code, there is a call to resp.code and resp.body. Twilio provides another value resp.moreinfo. This additional property of the response object is described in the response API. It will give your a more verbose explanation of why a REST request failed. I wound up finding out that my approved number was incorrect from this property (see previous paragraph) by tracing through the stack using rdebug. Again, familiarity with the lanaguge’s facets and debugging tools will make your race much easier. Unfamiliarity with moreinfo cost me time and thus functionality :-/.

Alan Davis, when not being harrassed on GChat by yours truly

As I was debugging that issue I had help from Alan Davis and Andrew Benton on the Twilio team via GChat. With a simple Twilio paging app completed, I was ready for a midday snack. After that I needed to put the logic of my stand-alone app into a web framework: both on my local MacBook Pro as well as a remote system reachable by Podio as well as Twilio, i.e. on Heroku.

Building the Middleware: Heroku and Sinatra

I started off small in Sinatra, basically cloning the Sinatra tutorial by Joe Yates with additional support from The Sinatra Book. This set me up with a small web service on my local machine as a proving ground for what I would deploy onto the real Internet on a Heroku node.

In about a half hour I had a Sinatra app running on my localhost that blended in the Twilio logic from the previous step. This file is podio.rb.

require 'sinatra'
require 'twiliolib'


# Twilio REST API version
API_VERSION = '2010-04-01'

# Twilio AccountSid and AuthToken
ACCOUNT_SID = 'XXXXXXX'
ACCOUNT_TOKEN = 'XXXXXXX'

# Outgoing Caller ID previously validated with Twilio
CALLER_ID = 'XXXXXXX';


before do
  # Great debugging!
  pp params
end

helpers do
  def send_a_page(rest,num)
      account = Twilio::RestAccount.new(ACCOUNT_SID, ACCOUNT_TOKEN)

      d = {
          'From' => CALLER_ID,
          'To' => rest[0].nil? ? 'XXX-XXX-XXXX' : rest[0],
          'Body' => "Emergency page"
      }

     resp =
     account.request("/#{API_VERSION}/Accounts/#{ACCOUNT_SID}/SMS/Messages",
                               'POST', d)
     puts "#### code: %s\nbody: %s" % [resp.code, resp.body]
   end


  end
end

get '/' do
  'Minimal Sinatra Hello World!'
end


get '/razzle' do
  'You have been razzled!'
end

get '/simplenotify/' do
 'Paging out simple'
 send_a_page [ nil, 'simple notify', nil, nil ]
end

As you can see, given my inexperience with Sinatra I kept things simple. The first routes I added were ‘/’ and ‘/razzle’ (Old joke from the Perl camel book…). Having tested those locally, I was sure I had the right approach. I then wrote /simplenotify to see if I could page myself. This basically harvested the previous CLI implementation and had me underway. I next needed to get this code up into Heroku.

Heroku

As I said before, Heroku is not difficult. But Heroku is not easy either. Heroku deployments have a number of dependencies on constituent technologies that are not easily understood unless you are already familiar with the requisite, supporting technologies preferably in a realm where you have used each of them independently where their dependencies don’t cloud the issue. Regrettably, it doesn’t seem like there is much documentation that helps take a developer through that process (although this post might fill that void).

That said, deploying to Heroku once you know these pieces is a snap, but before then it is daunting, and especially so under time pressure! For first timers, your time will principally be spent here (yes, I did just split that infinitive).

Let me try to simplify things as much as possible:

1. Follow the quickstart, steps 1-3:
2. Get Heroku on Ruby 1.9: heroku stack:migrate bamboo-mri-1.9.2 in your working directory
3. Edit code in podio.rb that worked on localhost, commit it to the local git repository
4. Execute step 4 and push to heroku; your code is now on Heroku!
5. Fail

When I started I didn’t understand two critical constituent technologies that are needed to make Heroku work. Here they are.

Bundler and Gemfile

Bundler allows you to specify gems your app needs (on local development) in a Gemfile. Heroku respects this definition set when it gets your application pushed to it. Therefore any gem you need should be specified in the Gemfile. Thus my Gemfile looks like:

source "http://rubygems.org"

gem "nokogiri"
gem "rack"
gem "sinatra", "1.2.6"
gem "twiliolib"
gem "podio"

What does this mean? It means that I want you, Heroku, to install nokogiri, rack, sinatra, twiliolib, podio into this instance’s (be it local or Heroku) gemset repository. On the local system, thanks to my .rvmrc this will ALSO be in the podio gemset I specified when I set up the workspace. Thus I can issue bundler install and the gems will be downloaded and, thanks to RVM, be stored in the podio gemset. This is very cool. When this directive goes to Heroku the same thing will happen. Heroku will grab all the gems specified in the Gemfile and store it in your app’s gem repository. If you want to require it, make sure its gem is loaded in the Gemfile. Speaking of require…

Rack and config.ru

Sinatra uses ‘rack’ as middleware. To configure how a Rack instance is to be launched, you specify the criteria in config.ru. Here’s mine:

require 'sinatra'
require 'twiliolib'
require 'pp'
require './podio.rb'

run Sinatra::Application

It is the logical complement to the Gemfile. It says, given those gems that are now available (thanks to bundler and Gemfile), I want to require these libraries into the namespace:

• sinatra: (obviously!)
• twiliolib: For use in Twilio operations
• pp: pretty-print debugging
• ./podio.rb: My code!

The last step is a command to run a Sinatra::Application.

With these two files in place and in the git repository, we’re good to run a Heroku-based simple-page out. With a simple git push heroku master I was able to get my code up on Heroku. The heroku logs (-n lineCount) command was a good friend during this process. Pointing a browser at http://my-heroku-instance.heroku.com/simplenotify sent my phone a page. Next was to tie the other half in: link into Podio.

Neil uses Skype to contact Podio home-base in Copenhagen for Andreas’ help and Journey playlist

Podio API

First, I went back to my app editor and added a field called “Emergency” with values of “Yes” and “No.” Then I saved the app. This would be my signal of whether to page out.

Next I clicked the “Settings > WebHooks” menu. I added a new hook and specified that item.create tasks should trigger the URL to my Heroku instance called /verify. This name is a bad name, but I was in too much of a hurry to change it. I should have called it something else. I specified, thus, a URL of the form http://my-heroku-instance/verify.

The next step is to verify your URL. To accomplish this Podio will send a form-encoded data blob (POST) to your URL. You must catch this and send back the validation information specified per the spec.

At this time I was encouraged by Christian to make use of the Ruby client API to handle the validation. Ergo I added podio to my Gemfile and required podio in my config.ru. I then ran bundler install and viola, I had the gem I needed. I then did another git push heroku master and Heroku built in the new dependency. I was now able to get validation working via API.

Getting validation set up took me a while, but it basically boils down to this.

In my podio.rb, in the listener for the route I specified in the WebHook configuration to use I added the following logic:

require 'sinatra'
require 'twiliolib'

post '/verify' do
  if params["type"] == "hook.verify"
    Podio.setup( :api_key => 'MY API KEY', :api_secret => 'SECRET_STUFF')
    Podio.client.authenticate_with_credentials('MY_PODIO_EMAIL_LOGIN', 'PW')
    Podio::Hook.validate params["hook_id"],  params["code"]
  end
end

Then you hit the “Verify” button in Podio. If you watch your Heroku logs, you’ll see the request come through and the validation trigger occur. NOTE: I lost a lot of time because I had ‘get’ as the first word for /verify. Verification happends as a POST operation. It was getting late at that point and I was completely blind to that error.

Probably about this point I got stuck on this stupid bug…

Now that the hook was validated, all I needed was a means to catch item.create operations. So I added:

  if params["type"] == "item.create"
   send_a_page [ nil, 'simple notify', nil, nil ], params["item_id"]
  end

This would pass the configuration message as well as the critical item_id for the newly-created item to a helper method which would implement the Twilio bridge to place a page.

Here’s that method, it should look familiar based on the content presented thus far.

helpers do
  def send_a_page(rest,num)

   Podio.setup( :api_key => 'XXXXXX', :api_secret => 'XXXXXX')
   Podio.client.authenticate_with_credentials('XXXXXX', 'XXXXX')

   # The ID passed in from /verify
   anItem=Podio::Item.find(num)

   # Very ugly JSON hash dereferencing
   theField=
     anItem["fields"].detect{|x| x["external_id"] == "is-this-an-emergency" }
   theRes=anItem["tasks"].first["created_by"]["name"]

   # If the emergency value field is set to use
   if  theField["values"].first["value"] =~ /yes/i
     # Cheap Heroku logging :)
     puts "YES!  It is an emergency"

      account = Twilio::RestAccount.new(ACCOUNT_SID, ACCOUNT_TOKEN)

      d = {
          'From' => CALLER_ID,
          'To' => rest[0].nil? ? 'XXX-XXX-XXXX' : rest[0],
          'Body' => "Emergency page: Task: #{params["item_id"]}" +
                          " from resident "+
                           "#{theRes} has been reported in Podio!"
      }

     resp =
     account.request("/#{API_VERSION}/Accounts/#{ACCOUNT_SID}/SMS/Messages",
                               'POST', d)
     puts "#### code: %s\nbody: %s" % [resp.code, resp.body]
   else
     puts "NO"
   end
  end
end

Finish Line

At this point all the pieces were in place and I created a test item, set to emergency, and had it page my cell phone. It worked! I finished about 10 minutes ahead of time and did something revolutionary.

I STOPPED WORKING

A failing app is a non-competitor, so I decided to quit with a small bit of functionality in place. I had planned for more, but learning Heroku, Sinatra, Twilio, and Podio had taken more time than I had expected. I credit knowing when to stop to be the result of 10 years hard experience in coding before deadlines or change request endings.

Comments

The story I tell here is pretty straight-forward and linear, but in reality (and I have the git commit logs to prove it!) it was much messier. I hope anyone interested can follow this post for didactic purposes not narratorial veracity.

Demo

I grabbed a beer and started talking with some of the guests who had come for the demos. The beer was necessary because my furious coding had left my nerves shot and my hands shaking. We sat down and were called upon to demonstrate in order. When my turn came I explained about the need for case tracking and alert systems. I created a new “Is It Done Yet” item, set the flag to emergency, and submitted the item. And as I talked I felt a buzz and ring from my chest pocket: I pulled out my phone and showed that the alert had worked!

I demonstrate…

It was a good feeling to get down from the presentation with a working app. The other entrants were all great. Alas, several good ideas were also presented where the developers couldn’t quite get all the pieces working in the time allotted.

Cardinal Blue did a great job

The Winner, Me?

When the judges’ delibrations had finished they returned and announced that 3rd prize was going to Nick who had worked with us remotely from Copenhagen (home of Podio). He won the sweet headphones as his application was designed to catalog entrants in the Eurovision contest.

Coming up next, for the iPad 2, was the team from Cardinal Blue. I thought their conference call bridging app (also using Twilio!) between Podio contacts was really great.

But I was very surprised when Phil Chambers announced that the winner was “Team Solo” that is, me. I was awarded the beautiful cinema display on which, even now, I type this document.

Checking my iPhone for an update as triggered from Podio for the camera

I gave a little speech, thanking the Twilio guys and the Podio guys who helped me. I was thankful to all of them for creating systems that had openness and hybridization at their heart. With such a good design principle at their core, all apps can be integrated for great and surprising effect.

After that it was time for more beer and a great lasagne dinner. Podio is first-class all the way through.

I had a great night talking with contacts who urged me to take the idea further and I hope to talk with all of them again soon.

Thanks

I am so thankful to everyone who helped me on the way and am thankful to Podio for their generosity and initiative in setting this up. I urge all of you to try to make a hack fest. It really gels your skills and, who knows, you might win!

• design
• productivity
• ruby
• san_francisco
• software

• Next: Gorgeous Ladytron Video
• Previous: SoMa Maps