On May 14^th, I competed at a Hackfest hosted by Podio. Podio is a customizable social networking application delivered as a service (aaS). After 8 hours of coding, I placed first in the competition and won a beautiful Apple Cinema Display. In this post I will cover my hack, how it was done, and lessons learned.

Victori pretium it

Or, you can watch my video interview

Background

Let me first introduce the various companies involved and technologies provided or implemented in the solution. I think of these as the ingredients in the recipe that allowed me to code my offering.

Podio

In case you haven’t noticed, people are spending phenomenally large amounts of time on social networks. These applications have their own vocabulary of interactions and provide certain defined workflows that, thanks to their popularity, are teaching the population certain workflows for successful collaboration. As younger employees enter the workforce, their models of collaboration will be informed by these social networks whose existence and interaction vocabulary are presumed to be ubiquitous. Accordingly, it would be wise for businesses to adjust their workflows to look something like these social sites.

The traditional solution to this has been to acquire servers, place them in a data center, and build your own social networking clone which, invariably, would be perceived as a dismal dog in comparison to the glittery product like Facebook, Linked In, et al. distilled in the market crucible. In short, building a social platform is probably not your core competency.

Enter Podio who offers a software solution that behaves “like Facebook” and which can be used as a service.

Traditionally, when using an aaS solution, one loses the ability to create highly specfic applications (“Podio’s not going to build an app to show what’s in our cafeteria today”). Or one might expect that to get a specific app one would have to pay Podio a handsome consulting engagement fee. Podio has seen a way around these drawbacks of using an aaS service by providing an in-application, friendly, web-based, drag-and-drop application builder. This is the first key element in my solution.

While a well-conceived application builder is great, certain behaviors are unique to certain businesses (e.g. “SMS the sales manager if no one takes ownership of the customer complaint within 24 hours.”) and don’t have an obvious integration point in a web interface. To remedy this concern, Podio provides a rich API (Application Program Interface) that’s a pleasure to use. It allows 3^rd parties to extend and share functionality between the Podio core and their own custom applications. It was the public release of this API on Saturday that occasioned the Hackfest. It was also the second key element in my solution.

On the topic of the API, let me add that many technical entities develop an API as an afterthought and the reliability and consideration (or lack thereof) shows. Podio bravely (!) decided to re-implement their web interface as a client of their own API. This is a brave move, but by testing whether they themselves could work with the API they were providing, they wound up creating a better, more-intuitive app.

Incidentally Podio is not purely a virtual experience, they have the Podio Store in San Francisco’s SoMa districit where you can go and talk to their staff about what you could do with Podio. They’ve made the “app store” software conceit physical. It’s revolutionarily transgressive in a good way

Twilio

Twilio is a service vendor who provides an exceedingly simple-to-use API for interacting with phones: setting up phone menus (“Press 1 for …, Press 2 for…”), making phone calls, bridging calls, setting up conference calls, setting up answer menus, etc. is a breeze. If you need to do something that touches a phone, Twilio’s rich, well-documented API will make it easy.

I had gotten a Twilio developer key and $30.00 of free minutes that I hadn’t touched since it was given to me at the Social Apps workshop held at Heroku in January 2010 (ulp). Twilio’s API was the third component in my solution.

Heroku

Heroku makes it incredibly easy to stand up and host web applications. I have only ever used it to deploy Rails and Sinatra applications. Heroku makes a very complex process managable, but it is not without some complexity. Heroku’s service was the fourth and final vended technology involved in my solution.

Previous Experience

It is fair to say that I probably spent the most of my time on the hack day trying to figure out the right way to develop with Heroku. However, if you read through this post and reference the implementation process, while not completely effortless, you will capture the value proposition posed by Heroku’s use.

Previously I had used Heroku to host my Verba Latina in Viis Ferrorum (Latin on Rails) application that I used in a failed bid to get into grad school (Cue violins and tears). As such I was familiar with their streamlined, git-based deployment design as well as with the concept that you must configure your app to have a configuration manifest so that Heroku builds the application the right way.

Last January Heroku and Twilio hosted an introduction to using Twilio and Heroku to create social, phone-based applications. Here a pattern emerged whose introduction was very important to my Podio solution:

1. Acquire a Heroku instance
2. Run a web framework (the lighter the better, for debugging purposes)
3. Use your instance to bridge communications between API’s
4. Violà, mashup

However at the Social Apps workshop, I didn’t really do this, instead I learned about some other Ruby tools described below. While the pattern would have been good to have had some experience with, the following tools were also essential to my success at Podio, so it was good that I spent time learning a bit about them.

Ruby Tooling

Sinatra

Sinatra is a very light language for describing a web server. You say: hey, when someone points to you with / at the end, send back some text generated by executing some code. It’s incredibly simple, incredibly clear and I had done a “hello Heroku” on it as part of the Social Apps get-together.

Ruby

Sinatra is written in Ruby, so, I’m glad I know Ruby.

RVM

RVM is a way to have multiple Ruby interpreters installed on a machine at the same time such that their various libraries and dependencies don’t interfere with one another. It also has a wonderful tool called “gemsets” which allow you to store all your Ruby gems (libraries for specific tasks) in a single sandbox so that you know what you have when it comes time to deploy.

git

Git is for revision control. More than once I screwed up a file and needed to get back to a working state quickly, specifically because the competition was timed. The ability to not have to do dozens of ‘undo’ operations was critical.

vim

A fast editor for editing text. ‘Nuff said.

Ingredient List

Having shown the background above, I had the following ingredients when I showed up:

1. Podio:
   1. In-application, web-based, drag-and-drop application builder
   2. API
2. Twilio
   1. API
   2. Free developer minutes from Social Apps workshop
3. Heroku
4. Sinatra
5. Other development tools

As of Saturday morning, I would rate my proficiency with Podio, Twilio, and Heroku, and Sinatra as being very low. It is a tribute to the designers of these systems that I was able to stitch them all together in just a few hours.

The Hack

Kick-Off

First things first I arrived at Podio’s beautiful storefront on 6^th street, SoMa, SF, CA, but 2 blocks from my house, at 8am and was treated to some great brain-food for breakfast. The interior is beautuful with Scandinavian sensibility and with several ukelele’s decorating the wall. The loft-style space is modern and tasteful and a great place to visit.

The ukelele wall

The morning started off with Phil Chambers (CTO, @filchambers) and Christian Holm (Developer, @hlmrn) showing off what the Podio drag and drop tool and API is capable of.

Christian demonstrates

Now here’s a bit of a sheepish confession I have to make: I hadn’t really even considered there was a competition. I thought it was going to be a meet and greet, code some, and get some tasty food and / or beverages…but then Phil mentioned that there were prizes for the best use of the API! To sweeten the deal with the spectre of avarice, Podio had brought in some great prizes: Dr. Dre headphones, an iPad2, and a beautiful Apple Cinema Display. I figured going home with headphones wouldn’t be bad for a day’s work. I half-flirted with maybe even getting the iPad but didn’t put much store by it. Besides, I had work to do.

Ideation

After the “Go!” bell was rung, I thought about something that had been on my mind. They day before I’d had a frustrating visit with my HOA. I found myself wishing that there were some way for us to keep better track of our “shared” maintenance tickets (handled via email). The first thing I did was use Podio’s drag-and-drop application builder to build a basic ticket tracker. While most tickets are private and bonded to a single resident, there should be “shared / public” tickets for shared issues: water stains in a hallway, Insanity Wolf in the garage, etc.

A fearsome menace! Insanity wolf does not read your Tumblr blog

I called this app “Is It Done Yet?” — a common-enough thought among most urban flat-dwellers when considering the state of their work orders with their property management companies.

By using the Podio app builder …

Click for larger view

I was able to create “IIDY?” tickets…

Click for larger view

This took me all of about 15 minutes to put together. It was about this time that Jon Froda, one of the founders, came by to talk to me about how things were going. I explained the work-order tracking idea and he suggested that maybe I could add an alerting system for “Emergency” requests and that I could have that page out. Having been a sysadmin, I knew about pager notifications and while I thought it might be a bit too simple to be a real competitor, I knew that it was implementable in a few hours…provided I use Heroku and Twilio. I was a bit daunted since I was not too familiar with these peer components and I certainly had no experience in tying them to the Podio API.

…But Jon’s idea seemed audacious to win a prize but was slim enough to do in a few hours. I went with it.

I thought the idea over and sketched up a basic schematic for the design with the help of Christian. I worked from right to left.

Podio                    Heroku-hosted Custom App               Twilio
 Web Hook On Create -->    Receives event                  |-->API sends SMS
                           Evaluates emergency Status      |
                           If emergency, paged via Twilio -|

Set Up the Development Space

$ mkdir podio
$ cd podio
$ git init
$ echo '\*.sw\*' >> .gitignore
$ touch Gemfile Gemfile.lock
$ echo 'rvm use ruby-1.9.2-p180@podio' >> .rvmrc
$ cd ~
$ rvm use ruby-1.9.2-p180
$ rvm gemset create podio
$ cd - 

info: Using ruby 1.9.2 p180 with gemset podio
$

As I mentioned earlier, knowing some core Ruby / Unix tools made getting started and working efficiently an edge for a team of one. These command set creates a directory, a primitive git ignore file, and creates a Gemfile that will be used for defining the apps’ gem dependenices via bundler as required by Heroku. It also specifies an RVM gemset which will sandbox all the dependencies.

Send an SMS via Twilio from the CLI

Twilio provides an Ruby wrapper for dealing with their API. The first thing I did was get the twiliolib gem. I then found the Twilio SMS example that, although written in PHP, was sufficient to give me a place to start.

I implemented this code like so:

#!/usr/bin/env ruby

require 'twiliolib' 
require 'pp'

# Twilio REST API version 
API_VERSION = '2010-04-01'

# Twilio AccountSid and AuthToken 
ACCOUNT_SID = 'MY-SID' 
ACCOUNT_TOKEN = 'MY-AUTH-TOKEN'

# Outgoing Caller ID previously validated with Twilio 
CALLER_ID = 'MY-VALIDATED CALLER ID';

# Create a Twilio REST account object using your Twilio account ID and token
account = Twilio::RestAccount.new(ACCOUNT_SID, ACCOUNT_TOKEN)

d = { 'From' => CALLER_ID, 'To' => 'MY-CELL-PHONE', 'Body' => ARGV.empty? ?
'An EMERGENCY severity task has been opened!  Please check PODIO task list.'
: ARGV[0] }

resp =
account.request("/#{API_VERSION}/Accounts/#{ACCOUNT_SID}/SMS/Messages",
'POST', d) resp.error! unless resp.kind\_of? Net::HTTPSuccess

puts "code: %s\nbody: %s" % [resp.code, resp.body]

What could be easier!? Well, I got killed by a bug here that took a lot of debugging. I had put in the wrong CALLER_ID value. Make sure you cut and paste the validated phone number value into your code. You can find the number on your Twilio user page.

Tip: In the last puts of the code, there is a call to resp.code and resp.body. Twilio provides another value resp.moreinfo. This additional property of the response object is described in the response API. It will give your a more verbose explanation of why a REST request failed. I wound up finding out that my approved number was incorrect from this property (see previous paragraph) by tracing through the stack using rdebug. Again, familiarity with the lanaguge’s facets and debugging tools will make your race much easier. Unfamiliarity with moreinfo cost me time and thus functionality :-/.

Alan Davis, when not being harrassed on GChat by yours truly

As I was debugging that issue I had help from Alan Davis and Andrew Benton on the Twilio team via GChat. With a simple Twilio paging app completed, I was ready for a midday snack. After that I needed to put the logic of my stand-alone app into a web framework: both on my local MacBook Pro as well as a remote system reachable by Podio as well as Twilio, i.e. on Heroku.

Building the Middleware: Heroku and Sinatra

I started off small in Sinatra, basically cloning the Sinatra tutorial by Joe Yates with additional support from The Sinatra Book. This set me up with a small web service on my local machine as a proving ground for what I would deploy onto the real Internet on a Heroku node.

In about a half hour I had a Sinatra app running on my localhost that blended in the Twilio logic from the previous step. This file is podio.rb.

require 'sinatra'
require 'twiliolib'


# Twilio REST API version
API_VERSION = '2010-04-01'

# Twilio AccountSid and AuthToken
ACCOUNT_SID = 'XXXXXXX'
ACCOUNT_TOKEN = 'XXXXXXX'

# Outgoing Caller ID previously validated with Twilio
CALLER_ID = 'XXXXXXX';


before do
  # Great debugging!
  pp params
end

helpers do
  def send_a_page(rest,num)
      account = Twilio::RestAccount.new(ACCOUNT_SID, ACCOUNT_TOKEN)

      d = {
          'From' => CALLER_ID,
          'To' => rest[0].nil? ? 'XXX-XXX-XXXX' : rest[0],
          'Body' => "Emergency page"
      }

     resp = 
     account.request("/#{API_VERSION}/Accounts/#{ACCOUNT_SID}/SMS/Messages",
                               'POST', d)
     puts "#### code: %s\nbody: %s" % [resp.code, resp.body]
   end


  end
end

get '/' do
  'Minimal Sinatra Hello World!'
end


get '/razzle' do
  'You have been razzled!'
end

get '/simplenotify/' do
 'Paging out simple'
 send_a_page [ nil, 'simple notify', nil, nil ] 
end

As you can see, given my inexperience with Sinatra I kept things simple. The first routes I added were ‘/’ and ‘/razzle’ (Old joke from the Perl camel book…). Having tested those locally, I was sure I had the right approach. I then wrote /simplenotify to see if I could page myself. This basically harvested the previous CLI implementation and had me underway. I next needed to get this code up into Heroku.

Heroku

As I said before, Heroku is not difficult. But Heroku is not easy either. Heroku deployments have a number of dependencies on constituent technologies that are not easily understood unless you are already familiar with the requisite, supporting technologies preferably in a realm where you have used each of them independently where their dependencies don’t cloud the issue. Regrettably, it doesn’t seem like there is much documentation that helps take a developer through that process (although this post might fill that void).

That said, deploying to Heroku once you know these pieces is a snap, but before then it is daunting, and especially so under time pressure! For first timers, your time will principally be spent here (yes, I did just split that infinitive).

Let me try to simplify things as much as possible:

1. Follow the quickstart, steps 1-3:
2. Get Heroku on Ruby 1.9: heroku stack:migrate bamboo-mri-1.9.2 in your working directory
3. Edit code in podio.rb that worked on localhost, commit it to the local git repository
4. Execute step 4 and push to heroku; your code is now on Heroku!
5. Fail

When I started I didn’t understand two critical constituent technologies that are needed to make Heroku work. Here they are.

Bundler and Gemfile

Bundler allows you to specify gems your app needs (on local development) in a Gemfile. Heroku respects this definition set when it gets your application pushed to it. Therefore any gem you need should be specified in the Gemfile. Thus my Gemfile looks like:

source "http://rubygems.org"

gem "nokogiri"
gem "rack"
gem "sinatra", "1.2.6"
gem "twiliolib"
gem "podio"

What does this mean? It means that I want you, Heroku, to install nokogiri, rack, sinatra, twiliolib, podio into this instance’s (be it local or Heroku) gemset repository. On the local system, thanks to my .rvmrc this will ALSO be in the podio gemset I specified when I set up the workspace. Thus I can issue bundler install and the gems will be downloaded and, thanks to RVM, be stored in the podio gemset. This is very cool. When this directive goes to Heroku the same thing will happen. Heroku will grab all the gems specified in the Gemfile and store it in your app’s gem repository. If you want to require it, make sure its gem is loaded in the Gemfile. Speaking of require…

Rack and config.ru

Sinatra uses ‘rack’ as middleware. To configure how a Rack instance is to be launched, you specify the criteria in config.ru. Here’s mine:

require 'sinatra'
require 'twiliolib'
require 'pp'
require './podio.rb'

run Sinatra::Application

It is the logical complement to the Gemfile. It says, given those gems that are now available (thanks to bundler and Gemfile), I want to require these libraries into the namespace:

• sinatra: (obviously!)
• twiliolib: For use in Twilio operations
• pp: pretty-print debugging
• ./podio.rb: My code!

The last step is a command to run a Sinatra::Application.

With these two files in place and in the git repository, we’re good to run a Heroku-based simple-page out. With a simple git push heroku master I was able to get my code up on Heroku. The heroku logs (-n lineCount) command was a good friend during this process. Pointing a browser at http://my-heroku-instance.heroku.com/simplenotify sent my phone a page. Next was to tie the other half in: link into Podio.

Neil uses Skype to contact Podio home-base in Copenhagen for Andreas’ help and Journey playlist

Podio API

First, I went back to my app editor and added a field called “Emergency” with values of “Yes” and “No.” Then I saved the app. This would be my signal of whether to page out.

Next I clicked the “Settings > WebHooks” menu. I added a new hook and specified that item.create tasks should trigger the URL to my Heroku instance called /verify. This name is a bad name, but I was in too much of a hurry to change it. I should have called it something else. I specified, thus, a URL of the form http://my-heroku-instance/verify.

The next step is to verify your URL. To accomplish this Podio will send a form-encoded data blob (POST) to your URL. You must catch this and send back the validation information specified per the spec.

At this time I was encouraged by Christian to make use of the Ruby client API to handle the validation. Ergo I added podio to my Gemfile and required podio in my config.ru. I then ran bundler install and viola, I had the gem I needed. I then did another git push heroku master and Heroku built in the new dependency. I was now able to get validation working via API.

Getting validation set up took me a while, but it basically boils down to this.

In my podio.rb, in the listener for the route I specified in the WebHook configuration to use I added the following logic:

require ‘sinatra’ require ‘twiliolib’

post ‘/verify’ do if params[“type”] == “hook.verify” Podio.setup( :api_key => ‘MY API KEY’, :api_secret => ‘SECRET_STUFF’)
Podio.client.authenticate_with_credentials(‘MY_PODIO_EMAIL_LOGIN’, ‘PW’) Podio::Hook.validate params[“hook_id”], params[“code”] end end

Then you hit the “Verify” button in Podio. If you watch your Heroku logs, you’ll see the request come through and the validation trigger occur. NOTE: I lost a lot of time because I had ‘get’ as the first word for /verify. Verification happends as a POST operation. It was getting late at that point and I was completely blind to that error.

Probably about this point I got stuck on this stupid bug…

Now that the hook was validated, all I needed was a means to catch item.create operations. So I added:

if params["type"] == "item.create"
 send_a_page [ nil, 'simple notify', nil, nil ], params["item_id"]
end

This would pass the configuration message as well as the critical item_id for the newly-created item to a helper method which would implement the Twilio bridge to place a page.

Here’s that method, it should look familiar based on the content presented thus far.

helpers do
  def send_a_page(rest,num)

   Podio.setup( :api_key => 'XXXXXX', :api_secret => 'XXXXXX')   
   Podio.client.authenticate_with_credentials('XXXXXX', 'XXXXX')

   # The ID passed in from /verify
   anItem=Podio::Item.find(num)

   # Very ugly JSON hash dereferencing
   theField=
     anItem["fields"].detect{|x| x["external_id"] == "is-this-an-emergency" }
   theRes=anItem["tasks"].first["created_by"]["name"]

   # If the emergency value field is set to use
   if  theField["values"].first["value"] =~ /yes/i 
     # Cheap Heroku logging  
     puts "YES!  It is an emergency" 

      account = Twilio::RestAccount.new(ACCOUNT_SID, ACCOUNT_TOKEN)

      d = {
          'From' => CALLER_ID,
          'To' => rest[0].nil? ? 'XXX-XXX-XXXX' : rest[0],
          'Body' => "Emergency page: Task: #{params["item_id"]}" +
                          " from resident "+ 
                           "#{theRes} has been reported in Podio!"
      }

     resp = 
     account.request("/#{API_VERSION}/Accounts/#{ACCOUNT_SID}/SMS/Messages", 
                               'POST', d)
     puts "#### code: %s\nbody: %s" % [resp.code, resp.body]
   else
     puts "NO"
   end
  end
end

Caleb invited me to show up via the SF Ruby group

Finish Line

At this point all the pieces were in place and I created a test item, set to emergency, and had it page my cell phone. It worked! I finished about 10 minutes ahead of time and did something revolutionary.

I STOPPED WORKING

A failing app is a non-competitor, so I decided to quit with a small bit of functionality in place. I had planned for more, but learning Heroku, Sinatra, Twilio, and Podio had taken more time than I had expected. I credit knowing when to stop to be the result of 10 years hard experience in coding before deadlines or change request endings.

Comments

The story I tell here is pretty straight-forward and linear, but in reality (and I have the git commit logs to prove it!) it was much messier. I hope anyone interested can follow this post for didactic purposes not narratorial veracity.

Demo

I grabbed a beer and started talking with some of the guests who had come for the demos. The beer was necessary because my furious coding had left my nerves shot and my hands shaking. We sat down and were called upon to demonstrate in order. When my turn came I explained about the need for case tracking and alert systems. I created a new “Is It Done Yet” item, set the flag to emergency, and submitted the item. And as I talked I felt a buzz and ring from my chest pocket: I pulled out my phone and showed that the alert had worked!

I demonstrate…

It was a good feeling to get down from the presentation with a working app. The other entrants were all great. Alas, several good ideas were also presented where the developers couldn’t quite get all the pieces working in the time allotted.

Cardinal Blue did a great job

The Winner, Me?

When the judges’ delibrations had finished they returned and announced that 3rd prize was going to Nick who had worked with us remotely from Copenhagen (home of Podio). He won the sweet headphones as his application was designed to catalog entrants in the Eurovision contest.

Coming up next, for the iPad 2, was the team from Cardinal Blue. I thought their conference call bridging app (also using Twilio!) between Podio contacts was really great.

But I was very surprised when Phil Chambers announced that the winner was “Team Solo” that is, me. I was awarded the beautiful cinema display on which, even now, I type this document.

Checking my iPhone for an update as triggered from Podio for the camera

I gave a little speech, thanking the Twilio guys and the Podio guys who helped me. I was thankful to all of them for creating systems that had openness and hybridization at their heart. With such a good design principle at their core, all apps can be integrated for great and surprising effect.

After that it was time for more beer and a great lasagne dinner. Podio is first-class all the way through.

I had a great night talking with contacts who urged me to take the idea further and I hope to talk with all of them again soon.

Thanks

I am so thankful to everyone who helped me on the way and am thankful to Podio for their generosity and initiative in setting this up. I urge all of you to try to make a hack fest. It really gels your skills and, who knows, you might win!

In other words, par for the course in the world of software development.

But then he was indicted, and convicted, for the murder of his wife amid a tale of S&M, Linux development ( intimately linked ), Russian internet-ordered brides, and infidelity.

A crucial feature of the trial was, well, that the cops couldn’t find the body. Upon being found guilty, Reiser seems to have copped a plea with the judge such that he could get a lesser sentence in exchange for the victim’s family and, nota bene, his own children being able to lay the body of their daughter / mother to rest.

Here was Gawker media’s “Valleywag” summation picture:

First of all, and not to be juvenile, but a copy-editor would have caught the phrase “fingers corpse”—oh right this is blogging, ahem, never-mind.

Secondly, the incapacitated girl in the ad in the party dress appears dead if not really whacked out on laudanum. One could foreseeably think that that was the corpse under discussion.

The whole post is pretty distasteful, I’d say.

I’m reminded of pro-feminist blogs decrying things like “new bikini’s are scandalous” or left-leaning blogs that decry “It’s insane that McCain can run for president in the 21^st century given that he can’t even use a computer” only to be served up, guess what, an ad featuring said bikini’s or a clip of the blogger-hating senator as an ad.

In the immortal words of Sid Meier’s “Civilization”: “Time Passes…”

1. \xC4\x81 is the UTF-8 encoding for the Unicode code point U+0101.

2. [Q:] Which table does U+0101 fall into?
   
   [A:] “So the first 128 characters (US-ASCII) need one byte. The next 1920 characters need two bytes to encode. This includes Latin alphabet characters with diacritics, Greek, Cyrillic, Coptic, Armenian, Hebrew, and Arabic characters. The rest of the BMP characters use three bytes, and additional characters are encoded in four bytes.”

3. OK this means that the code point will be of the form: “110yyyyy 10zzzzzz”

4. We will now work to fill in the “y” and “z” values:

5. Hexidecimal “U+0101” converts to binary: “100000001”

6. There are 5 y’s and 6 z’s. So let’s split the above number to match that form: “[00]100-000001”. Note, we moved from the right. Where the leading 0’s were required to turn 100 into 00100, they were pre-pended.

7. Integrate and produce: “110” + “00100” : “10” + “000001” => 11000100 : 10000001

8. Take THESE numbers and convert them back to hex => c4 81

9. String notation for this is \xc4\x81 - violá!

10. Figuring this out letter by letter is a major pain in the keester. A good URL resource is: Fileformat.info or, handily a URI of the form:

http://www.fileformat.info/info/unicode/char/<unicode char value>/index.htm

Special Thanks to: Michael Flester

I’ve not written for so long owing to my fear of having to migrate more data or lose more data should the lost stevengharms.com die that I’ve completely fallen out of the habit of writing.

So far the folks at hostgator have meen a dream to work with, I opened 3 different support cases with respect to getting things set up and going and all three were addressed and dealt with between the hourse of 9 and 5. Totally, professional, friendly, and smooth.

I noticed a few links on the navigation tab are broken, i’ll be getting to those shortly, but for the moment my virtual home is safe.

First, let’s just say that it’s not the expectation of the teacher or the class curriculum that you work yourself into fatigue of this type. It’s not necessary or required. Yet in both of the BNR classes I’ve taken, the students worked late into the night on their own projects, or improving the assignments.

Therefore, the motivation to work to this level of fatigue is not extrinsic, it is clearly intrinsic. This is, clearly, the more cruel mistress. In a previous post I spoke of the self-selection implicit in defining the population for BNR classes. I suggested that one of the primary contributing factors in being in the class is having the funds to pay the fee or having a job where the administrators feel that paying such a fee for such a person is a worthy investment. People who have such funds ( if the meritocracy be believed ) or faith of their financial administrators likely get to this position be not being able to simply let it go.

I have worked through weekends at work, on work stuff simply because I could not let it go. I have gone to work early, repeatedly, after a few hours of sleep because the problem stalked me. It wouldn’t let me go, it wouldn’t let me sleep. It was better to be awake, approaching understanding, than to endure fitful sleep under the Damocles sword of the unfound answer’s shadow.[1]

Many people I’ve talked to say: “Go do something else, come back to it.” My sweet girlfriend has shuffled around in the wee hours to find me tapping away and urged me to come back to bed. I hear this, but this is something that I ( and I suspect many others ) simply cannot do. If I were to come back to bed I would imagine my text editor. If I were to go do something else I would be irked that I couldn’t look up on Google possible solutions as I was doing it.

I don’t know it well enough. I don’t understand it well enough, I don’t see how the parts work. Granted this quality among the full rank of attendees, you can see how I would suggest that a startup out of any one of these classes could blow the doors off of any particular problem.[2]

So it’s under this background mentality that you’ll see BNR-class attendees whipping themselves to perform, to create, and realize until deep into the night. Simply put, this is just how we’re wired.

Attendees like to maximize effort. That’s why we’re programmers, we like the force multiplier effect of knowing a programming tool. The reason we’re at this class is because we want help learning something very difficult in the most efficient and deep manner possible. BNR classes deliver this. So much of the work done here can be seen in light of trying to optimize knowledge gained per unit time.

Most of us have attempted to learn the material back in “the real world”, but we couldn’t get the information to ‘etch’ deeply enough. Learning this material against deadlines, users, wives, taxes, other obligations was simply too slow or too confusing. If you are a “can’t let it go” person who knows that shortly he’ll have to be back in the real world where he’ll have to let it go and you won’t have access to immediate peer feedback or a guru, you work hard.

Like summer camp, we come, we know it will be hard, but we know, looking back, we’ll be amazed by how much we learned, how much we changed, and we’ll remember the conversations, the food, the hard nights. We’ll remember that we were part of a special community for a brief flicker of time.

And if, to chase to that next eureka, we have to have bleary eyes and catnaps in the conference room, so be it. Damn the torpedoes, I came to learn rails.

So we slurp down cups of rocket fuel, we dose up on small quick munchies, we talk, we think, we code, we edit, we close our eyes and doze and then turn back to it, we take more fuel[3]. At the end of a multi-hour run your sugar crash comes but you have to keep pushing. And sure, you could go back to your bedroom, but the feedback loop outside of Oz is too slow. You see, we have to be good enough to be independent when we leave because, it all likelihood, we will be.

So what’s it like, in the dark of night as we ply away? Well, to set the scene, we’re in a very large conference room with a large projection screen hanging from the front wall. The churn of the heating provides a constant white noise that is overlain with collective symphony of keyboard tapping[4] and occasional double-click.

For the most part it’s fairly quiet, except that there are ‘flexible duos’ merging and separating. These duos emerge when one party wants to discuss an idea or a problem he is describing to the other party. They conversation is usually pretty quiet, but occasionally precipitates a whiteboard session.

The lighting is dim, but not dark. The eyes of the coders in troglodyte mode is respected.

And, of course the half audible curse of ‘fuck’ or ‘damn’ as an execution fails, or even worse, does something you didn’t know about and or didn’t plan for.

The tablecloths that drape the tables are rumpled. There are big reference books from O’Reilly press and the Pragmatic Programmers stacked up, splayed open, or fallen onto the floor.

With the dim lighting and the lack of a clock in the room, Vegas-style, you could easily fall through scheduled reality and forget that time is actually passing. Eventually, the time will turn late and that’s when the real fatigue sets in.

There are enough peers present such that your quorum sensing lets you know “It’s OK to stay a bit longer”. Your eyes are dried out from staring at the screen, the typing fingers are sore and tired. The myriad of files you edit get lost, your new enemies are questions like “Where did I just write that thing a second ago?”

You reach the next item on your “to do” list and you attack that problem…and keep on going. In some ways it’s actually invigorating, working on something you care about this much, something that has this much of your love in it. Few things “outside” can hold your interest like this. In some ways, you feel more alive doing this kind of work in this environment that you may have felt at your day job for the last year.

Eventually the discussions between students achieves a sort of level of conversational shorthand. We’ve all been speaking to one another in these highly specialized terms for days and days. We’re all coming from the same reference and our discussions are short, brief, and truncated.

As the evening wears down more stammers work their way into the dialog. The solutions wear down. The bowl full of ice chips melts and the bottles, coffee mugs, and cans on the desk continue to multiply as the diminishing returns on coffee and sugar is reached, crossed, and run over by a distance of several miles.

But eventually you reach your goal and you shuffle back to your hotel room and unwind by watching infomericals or NOVA and then drift off to sleep.

This is the society of the hack, and you’ve done a good day’s work. Sleep is quick and hurried, because the next round of class starts bright and early at nine in the morning.

Footnotes:

1. Incidentally, the Silicon Valley exacerbates this problem. Most people can’t let it go, and there’s not much else to do besides work anyway, thus so many good ideas and good companies coming from that area.

2. This is what the early internet adopters must have thought would happen. Global, flexible, talented groups meeting and fluxing, producing brilliance. Regrettably, it seems that this talent needs to be pulled out of their quotidian affairs to achieve this maximal “group flow” dynamic ( see: Cocoa Dev House or Burning Man )

3. I recall a passage in The Fountainhead where Roark falls asleep at his drafting table and breaks a pot of coffee. I recall Ender Wiggin facing yet another round in the battle room after another brilliant performance around the absurd rules. In these passages I always see the author comme createur writing his experiences into the character’s life.

4. Not really tapping. Apple laptops don’t have a cheap, plastic, clackity sound, like the PCs, but rather have a very pleasing whisper.

After the lengthy tour of duty yesterday, I’m back in for another lengthy day. I think that Wednesday is really the grinder day.

First, you’ve been lectured to for the last 3 days. There are very few minds that can take 8hr. / day lectures without feeling a bit tapioca-ish on the 24th hour.

Secondly, you’re listening and then working very hard on something that you just were introduced to. You have these head bashing against a wall sessions and then, suddenly, you’re set free ( because you’re doing something that’s old hat, something you learned, say, that morning ) and then you hit the wall with all that built-up acceleration. It takes a certain mental flexibility.

Third, the sugar, the munchies, the food, etc. By this time it’s all having some funky effects on your body.

Homesickness becomes a bit of a factor. You remember how good your bed is, how nice it is to be among your things and your life.

Now, about the work. You see, you can slide on day 1. You learn the material, you review it, you’re probably OK. DON’T BE FOOLED this is your one and only chance to get ahead of the abbatoir days 2 and 3.

Then comes day 2. If you want to keep ahead you’ll be working late on day 2, maybe even thinking that you’re going to catch a break on day 3. But no, my friends, day 3 is the grinder. Day 3 says “OK, you’re determined, but how determined are you?” You’re bleary from day 2, the cokes, the mountain of water bottles, can you do another 18-hour run at things?

At day 4 you’re going to be exhausted. You’re going to reach your limit. You start to feel like your treading water capability is turning into negative progress. Unless you’ve stayed up on the exercises and mentally integrated the concept and become with the ‘way things are done in this language’, you’re going to feel a bit behind.

Fortunately day 5 is the wind-up day. There’s a sense of graduation and possibility. If you can integrate the first 3 days quickly you’re going to be in a good position to re-discover the last two days quickly. I think that may be the optimal take-away from a class like this.

I’m not sure, I’ll be a good empirical guinea pig for the next few months.

I had a late morning this morning … because I had a late night last night.

Last night, per BNR tradition ( and I can say this because of me experiences at the Cocoa camp ), post dinner ( and occasionally a drink or two ) many of the students returned to the classroom to review, invent, create, etc. under the tutelage of our teacher.

I basically re-ran all the exercises from Day 2 again, inventing scenarios to make sure that I truly “got it”. In training, it’s very easy to simply nod and do the exercise but, as my math teacher says, when you are working the problem alone, you will get the loneliness. You will say, “Where is my teacher? Where is the format, I’m so lonely, I cannot solve this problem.”

So, to make sure that I could work through the loneliness me and a few other students were here until about 1:30…and I wasn’t the last to leave! We Big Nerds be hard^core.

I moseyed into class about 5 minutes behind and Charles had already begun the lecture on “Data Validation”. It’s a pretty gentle way of starting the morning.

Outside, Georgia has tuned cold and windy, a lot like the Bay actually.

This morning we did a lesson data validation and man, I hit a tough bug. Rails was telling that the error was at some line in some process. With the help of another patient peer we commented out all the lines of my code and methodically worked through and found out that the error message was completely un-helpful, the error was some 6 lines earlier!

BNR classes always leave me with the feeling that working should be more like this: patient people focused on exploration and the delivery of solid products with understanding produced as a natural side-effect (oh yeah, and having an absolute guru around to ask questions to is not a bad part either). I’m pretty sure you could take any BNR class, tell them to attack some market problem, and they would be able to deliver it as a successful startup.

Consider the self-selection principle at play. People pay ( or are in positions where employers are willing to pay ) what, on paper, might seem like a rather steep fee ( in reality, it’s entirely reasonable for the type, depth, and knowledge provided ), this indicates a certain level of success obtained in the world. Further, even people of that level of success are usually rather loath to take one-fifty-second of their life and spend it doing “work” for at least 9 hours a day. Paul Graham said that the magic of a startup wasn’t that it was a small amount of people, it was a small amount of people like you. During these sessions you feel like you’re part of a larger community of people like you.

For 5 days people like you, focused on learning, learn, work and talk together. It’s kinda special. Sorta like camp, but instead of popsicle stick birdhouses you build powerful computer applications. If, in 5 days, we could do something like this, in 90 days what sort of application could we build?

I have not left the gravitational field of this big, blue, glob. There’s a bunch of interesting work stuff going on ( more later ), my mom was in town and I started classes at Austin Community College.

I’m taking two classes: Intermediate Algebra and C++.

First, this is the kind of mathematics I learned in high school ( or should have learned better ). I eventually matriculated to the university and there went as far as 2 semesters of business calculus: integration, derivation, the whole shebang. But, after ten ( and I am astounded to type that ) years away from calculus, I’ve forgotten so much. So, here I am, back at square one, learning the basics again. It’s easy to forget a lot because, to answer those kids in high school who asked “when am I going to use this in the real world” the answer, I’m afraid to say, is rarely. So much, that, you’re right, you might be wasting your time. Sorry.

In any case, this time through I’m finding it much easier to learn and encode this information. I’ve thought about why, but I think that my brain must have been conditioned for understanding symbolic and abstract systems through years of programming and a bit of symbolic logic. As is suggested by the action of Snow Crash, I think that the brain arrives with just a tiny bit of software pre-installed. The first several routines (“primary routines”) decide whether or not you will be more or less receptive to new (“secondary”) routines.

The primary routines must be incredibly fundamental. Do you use symbolic language, pictorgrams, pictographs? Using pictographs may disincline a learner from picking up a certain set of secondary routines ( I don’t believe anything is un-learnable, although research shows that past the age of 7 there’s no chance for language acquisition if it hasn’t already happened, sorry Tarzan).

So this secondary routine, algebra, just didn’t stick for me. But I think that I’ve been running secondary programs of an abstracting and variant nature now for so many years that receiving new routines which share similar pathways as other abstracting secondary routines’ function has made it easier to integrate the data.

Or, maybe because my teacher is exceedingly competent. Gustavo Cepparo is one of the best lecturers I’ve had at any school I’ve ever attended. He does not advocate “niceness” he educates giving you an education (although he is very personable). He sees himself as your worthy adversary, trying your skills and, in so doing, giving you an education. Damn straight.

I was discussing this “inclination for symbolic systems” with Lauren who, in her own life, is also undertaking an effort to put some ‘new secondary routine momentum’ into her gray matter. She’s learning computer science and programming. Oddly, when she programs she feels the same structure and scalpel that she came to know doing her literature work: themes, repetition, motifs, it’s all there.

But then we came to a question: Why is it, if you want to go into programming, pedestrian ( or parental ) wisdom holds: “Are you good at math.”

This, parents, friends, teachers, I want to warn you away from asking. The question is not “are you good at math” but do you like symbolic systems, creating them, imagining them, adhering and bending them? That’s the question.

Good programmers are kids who memorize the armor charts to Dungeons and Dragons. Good programmers like to corollate data on baseball cards, they like to organize baseball cards. I’ve seen kids read chess books, or play Magic or play Yu-Gi-Oh, know the Dewy Decimal system, it’s all the same thing: breathing life into internal rules processing machines in your brain, and then using physical ( versus digital ) objects as the inputs to your automata.

So why do kids get asked “but are you good at math”.

Math is a convenient, albeit misleading, question, it’s a forced symbolic system that kids are forced to learn. To this extent, it can be used as a good measure of “will you be a good programmer”. Further, and a child has no way of knowing this, this lazy question implies that “liking math” and “liking the pedagogical approach used by the school board and as practiced by your teacher” are the same. They are wholly different and a child has no way of knowing this.

I disliked most of my math teachers, and the pedagogy was slow, pedantic, too slow to build connections, to sketch an architecture, to paint a direction. Math class, for me, was about exercises and who in their right mind gives a flip about that? I knew math was important, but there was no direction or structure for that statement beyond the obscure “but you’ll need it in college”.

Here’s how I now propose that math should be explained.

• There are many difficult problems in the world ( how to get a satellite in a crater on the moon), how long to incubate a new medicine, etc.
• The language for expressing these ideas is mathematics. Just as anyone would look at this figure ( draw a capital ‘A’ ) without knowing the alphabet would suppose it’s a picture of a bird or an interesting shape, you must come to learn the basic letters and words of mathematics
• For the next several years, you will be learning basic ideas and words in mathematics, this study is called arithmetic.
• (later) You have learned arithmetic, but most questions in life do not hinge upon known quantities. Like we said in point 1, how do we do something that we don’t know, how do we model and predict? The branch of mathematics which deals with discovering unknown players in systems is called algebra
• (later) You have learned algebra. Algebra helps discover nouns in systems (what plus 4 equals 11), but the world is a constant state of flux, as noted by Newton. A mathematical language for discussing flux and rates of change was invented by him and Leibniz, that is called the calculus.

I don’t know much more about math than that, but with that framework I could have seen that learning that 3/4 * 4/3 = 1 was something important in the sense that it was as fundamental as learning the crucial verbs to express want or need or identity.

And that, balance, unknown, systems for discovery of unknown, systems for modeling the unknown is incredibly interesting. Had mathematics pedagogy been about systems of symbolic manipulation to discern the unknown versus timed tests and a litany of rules and obscure little “tricks”: loose islands of thought, I feel I would have grasped the beauty of math earlier.

And ultimately this brings me back to the most sublime poem ever written “Ode on a Grecian Urn”

Than ours, a friend to man, to whom thou say’st, “Beauty is truth, truth beauty”—-that is all Ye know on earth, and all ye need to know.

Algerbra, comes from the original Arabic book in which its tenets were first set: Science of the Reunion and the Opposition.

Isn’t that a beautiful phrasing for what algebra allows us to do? This is truth and this is beauty.

And hoary old double-entry accounting, it is a science of reuniting the not with the present, the received with the owed. That is truth, and that, to my good accountant friends, is beauty too. I can see why those who ply this craft love it.

And programming, it is the balancing of the abstract concepts against the abstract concepts. In this vacuum, you create, and you create function, and then you create beauty. And that’s why we love it.

And assuredly, if you choose to plumb the deepest depths of that digital reality, you will do a lot of math, but don’t scare off a child from playing in our world of pure abstraction because they mistakenly associate it with the pitiful educational quality in this country or rote, staid, pedagogy.

**I’m posting this, but I feel it’s incomplete, I think I need to read it some more and refine it, so, this content may update. **

A similar cult: WordStar