The question is often enough asked: “Why is there such a dearth of diversity in open source?” While some have supposed that there may be some sort of genetic difference (hogwash) or similar, this last week I had some experiences that showed me why newcomers to technical and open source communities might decide to quite quickly spin on their toes and walk right back out.
New US Industries
A reader recently sent me a nice bit of research about job market for MBAs. It was a well-written article and I enjoyed reading it. Readers, you may find this interesting as well. One of the sections marked for growth is “Big Data:” a topic near and dear to my heart.
I especially found this quote by Robert Fitzgerald of the Lorenzi Group interesting:
…take statistics and economics…[don’t] be choosy about…[their]…first job.
Fitzgerald was later quoted:
“Get ANY role you can. Yes, it will suck. Yes, you are worth more, or at least your degree is. And while you are in that role, in your free time, start analyzing everything. Come up with infographics that are funny, helpful, and interesting. Show them around the office. You will get noticed,” he said.
The quote seems to certify that the days of being a special snowflake by virtue of your degree or the debt you went in to attain it are over. Once upon a time a man (yeah, a man) could spend a few years at college, refine his Latin and Theology training, and then study for the bar, apprentice, become a lawyer and then a politician (cf. Noah Webster, John Adams, John Quincy Adams, et al).
MBAs in the year 2013 anno domini, I have bad news: labor is global and the paper with the blackletter repeating the name your parents gave you is worth far less than the fact that some guy in Asia knows books full of math, economics, and statistics and can offer not merely the promise of capability, but can deliver results right now.
Heck, this fellow beat out more people just to get his shot to learn those topics than you’ve ever competed against in your lifetime. You are at a disadvantage.
Potentially, as you move up the ladder (surely thanks to your witty infographics, admirable golf handicap, and hard work), you will enter a part of the influence and leadership curve where your MBA will help differentiate you from the database wizard: you’ve learnt “articulating vision” and have the shibboleth credentials of looking like executive management and have the good fortune of citizenship.
But opporunities for differentiation won’t come until later and you’ve got to service your debt and your household in the here and now. I liked Fitzgerald’s practical suggestion. It’s also a fact about the world of today: get used to running all the time.
Spring Break and Tech Conferences
What is it about tech conferences that makes individuals who work in a highly cerebral and reflective industry like software development go quite so nuts? Why the oral sex jokes, the thong-model backgrounds on slides, etc.? And why are the sponsors so willing to bring out Hooters girls or hire go-go dancers at their events?[1] In short, why does the tech conference misbehavior look so much like the hormonal, puerile misbehavior associated with Spring Break misbehavior? I believe that a contributing force rarely mentioned is the way conventions and their host cities (intentionally or not, tacitly or not) market the host city’s “vice” industries as a selling point[2].

It comes down to capitalism and insecurity. The cities that build convention centers and hotel complexes court conventions to attract conventioneers and their filthy lucre. It’s like shooting fish in a barrel: conventioneers, unfamiliar with the city, without a car and and lacking their standard social support mechanisms are in the perfect mindset to dispatch their insecurities with expensive calorie-bomb comfort food and alcohol. Inflating local business revenues and procuring the tax-runoff thereof by attracting these hordes must seem like such an obvious win politically and financially. It’s no surprise that municipalities and their politicians so readily reach for the “build a convention district” plan for municipal funds recovery.
I further contend that there are a few cities that have acquired the reputation of being cities with a really good vice market: San Francisco (North Beach, Tenderloin), New Orleans (French Quarter), Miami (Everywhere?), and the 2 ton gorilla, Las Vegas (Everywhere!). When you go to a convention in one of the cities there’s the unstated “Yeah it’s in $VICE_CITY … nudge-nudge-wink-wink.”
It’s probably no surprise therefore that the picture atop this post was a bit of litter I found on the sidewalk a block away from San Francisco’s Moscone convention center.
Models of Computer Science’s Science: Formal Versus Empirical
Introduction
On my walk home from work I often listen to the Ruby Rogues podcast. In their episode with Glenn Vanderburg on ”Complexity,” there’s a fascinating discussion on the nature of science, and how that defines the modus operandi of “computer science.” Dave, a panelist, asserts that: “computer science is heaving with science, but its heaving with formal science as opposed to empirical science.” I believe development is “heaving” with “formal science” because of of an all-too-human reason: vanity. Developers esteem formal reasoning over empirical reasoning out of peer pressure to seem “smart” which is associated more with the formal style of reasoning. I’d like to look at why this is the case.
Formal Correctness Versus Empirical Correctness
Given a set of propositions, ideas, lemmas, rules, etc., one can manipulate the input data, via a series of actions and “prove” that the system is “correct.” Much like logical truth, more helpfully called consistency in logic, this “correct” is not the “correct” that tests or use by end-users illustrates. Rather, this consistency or correctness follows from the form of the argument and the definition of the atoms and their transformative processes. The computer scientist Knuth’s humorous statement is recalled: “Beware of bugs in the following code. I have merely proved it correct. I have not tested it.”
This is the kind of “computer science” that can be done with a pencil and paper. It looks a lot like math or symbolic logic ergo formal science. It feels old-school, punch card, ur-nerd, Rand Corp, programs in 16K of memory badass.
As mentioned above, “correct” for end-users has nothing to do with the aforementioned formal correctness per se but translates to “the app did what I expected to. It didn’t blow up.” Like an 8-year-old discovering the miracle of baking soda and vinegar, empirical correctness is based on “did that blow up in the way I expected or not?”
This is the kind of “computer science” that requires a compiler and a unit test framework. It benefits from more computer tooling that makes the feedback loop between failing test, passing test, beautiful code being sped up. It doesn’t look like that staid horn-rims and tie computer science, it looks like SoMa and people command-tabbing through screens like monkeys undergoing attention defeceit research.
Developers’ Esteeming Formal Versus Empirical Correctness
I find that developers seem to have a natural, per-person bias towards empirical or formal testing of their code and that there is a privileged association with formal reasoning about code. Dave, speaking of his clients said:
Yeah. I’m actually facing with multiple clients recently this resistance to [empirical] testing. And there’s just this innate need to think about code and to logic your way to the facts.
Dave, I believe, used “innate” hyperbolically, I believe he really meant that “developers see empirical testing as less prestigious, B-team and don’t want to admit that they could benefit from it.”
Let’s imagine a bug crops up in a system, who wouldn’t like be the formal adept in this scenario:
You approach debugging from a mathematical point of view: proceeding from inputs what must be the case. You look at the code and fold you hands and, with the help of a single console-logging statement or debugger statement (or two) you find where in the logical exercise that things go the wrong way. You edit an if/else and the bug disappears.
Who wouldn’t like to be thought of like that? It makes you sound wicked smart. Alternatively:
You look at the bug report and see that the inputs were X, Y and 2. You know that the code blew up at line 22 so you put a debugger on line 21. You run the code with those inputs and look around the system. You manually make the deadly code on line 22 run, you evaluate its results. You see the bug. You write a test case to run the deadly input set. You run the test over and over (automatedly) while you fix the bug. You see the bug is fixed (via the test) you check in your test and your patched code and move on.
Doesn’t this sound less ivory tower, less ethereal, more plebeian? So if a consultant came in and told you in a meeting with your peers that your team should embrace automated testing would you be the one who would be brave enough to break the culture rank to say “Y’know, that’s a good idea” when the implicit cultural tenet is: “The best of us are smart enough to always logic our way through?”
In computer science, full of its math envy as it is, we want to look like those early mathematicians turned computer scientist like von Neumann, Turing and Adm. Hopper - formalists.
Out of fear of appearing like the weak one (lower peer/boss review, less pay, less respect, less prestige), the emperically-biased programmer stifles her dissent (something humans have a hard time with anyway) and presents, if not feigns, dislike of empirical methods. Perhaps this behavior is so universal that it seems “innate,” as Dave related.
It’s vanity and peer-pressure: the same forces behind most bad decisions in human history.
Conclusion
I think that all developers should be strong in both approaches: As an empiricist, I’ve worked hard to get better at formal evaluation and at writing code that’s easier to reason about so that I don’t have to step through it or make it blow up several times to figuring it out (empirical methods). However, I think we should all have the humility to admit when an empirical test helps figure out the bug.
As Kernighan said in “Unix for Beginners” (1979):
The most effective debugging tool is still careful thought, coupled with judiciously placed print statements.
Even the masters need visual confirmation sometimes!
Coda: Honest Talk
I feel like I would be dishonest if I were to not explain my own background on this question. In part I hope that if you read the previous section and thought “Wow, thanks for saying that” you might enjoy my being honest with you about how I mix the two disciplines. Here it is:
I didn’t come to development via computer science but by a mongrel polymath background so it’s less (just barely) embarrassing for me to say that I like empirical debugging. It feels good to me to make stuff blow up on command and then to know that when I’m done I’ve left a fix and a test behind, a check on other things blowing up in the future if someone else changes a part of the system. I like the emergent phenomenon that several small tests, in aggregate, might serve to cover more uses and possible error states than we had thought of by following the formal structure of the code.
But that comes with a catch: if you look at code and errors this way you may cover only one case of a whole variety of bug cases, you’re only being guided by the latest disaster. Eventually you’ll stop the bleeding but by layering a thousand bandages versus one perfectly placed suture. What to do? Here are the steps I follow:
Practical Debugging Steps
I found an interesting discussion on this topic by Jeff Vroom who said use the one to focus the other (formal to empirical and vice versa). I think this is good advice. Here’s what I, an empirically-biased developer, try to do to engage that “enhance the other” virtuous cycle:
- Something blew up
- Where?
- Generate a test case where those same inputs cause the blow up to happen. We future-proof against this error. Just as I don’t need accessibility features (big print, ramps, handicap door openers) I’m glad they’re there when I’m tired or my hands are full. Even the most brilliant formal thinkers will eventually be thankful for your empirical test
- Why did it blow up? I switch to formal analysis here
- Which of the inputs was invalid?
- How did it get set incorrectly?
- If you have code that adheres to the single responsibility principle and is nicely structured so that it has small parts you should be able to reason through the process easily
- If you need to, add a logger or debugger breakpoint to “get” the failure.
- If this code is hard to reason about promise to make it easier to reason about after you patch this bug
- Fix the failure
- Switch back to empiricism: is the test passing?
- Fulfill the promise: was it hard to reason about? Can we write the code in a way such that things are easier to reason about? This is refactoring. There are many good books on the topic and you only get better by practice
- Commit and move on
These are the steps by which I’m trying to make complexity more graspable for humans of all reasoning biases more quickly. Sometimes I do better than others, but there’s where I’m at today.
A Beginner Again: Rails 4.0 + Ruby 2.0 on Ubuntu VPS
I’ve been programming Rails for a while now, and with the major releases I feel like I’m back to square one. There’s new gotchas, new strangeness talking to the databse, etc. As I encounter strangeness, I’m documenting it here.
I started by installing rbenv and Ruby 2.0. I then followed this Stack Overflow post on how to install edge Rails. I really like this approach because it makes use of Git submodules.
Things after I got Edge rails installed, I then set up my Gemfile. I don’t want CoffeeScript so I handled that by commenting out its line in the Gemfile.
I installed Postgres via apt and that went swimmingly. I was suposed
to merely configure the config/databases.yml file and then rake
db:create my way to happiness. Here’s where the pain appeared. I kept
encountering an error like: FATAL: Peer authentication failed for
user.
Fortunately Stephen Goss at Delicious Robots Blog had the solution:
reconfigure pg_hba.conf. With that I was able to execute rake
db:create and see my databases. I’m planning on following Michael Hartl’s excellent tutorial mostly as a means to make sure that I step
through many of the new ways of doing things.
I’m planning on blogging any other snags I find along the way, but that Postgres one was a real nasty one.
Never Too Late
I have completely been remiss in terms of medical care. In fact, there’s a critical reason why: I was afraid. You see, in my last visit to a doctor ( about 7 years ago ) I got that kind of news nobody likes to hear: you have too much stress, you weigh too much, and your triglyceride levels look to be on the low end of acceptable.
After that I moved and never found a doctor I liked, moved again and never found a doctor that I liked and just kept ignoring that I needed to get a physical. I think it had been 6 years.
So the other week I resolved to remedy this situation and to talk to some friends about what sort of medical care they were receiving. I received a recommendation for One Medical here in San Francisco. It’s a club, basically, that one pays to join ($150/annum) and, in exchange, you get things like guaranteed same-day appointments, iPhone app appointments, medical interaction over 21st century media (email your results, automated reminders on your GMail calendar, etc.) — you get the “e-Medicine” promise that seem, largely, to have never delivered on its vision.
Today, as I read my mail via iPhone on Muni, I received a summary:
1 2 3 4 5 6 | |
Somewhere over the last 7 years I got healthier. I attribute it to the following:
- Living with a Southern Californian, their endemic “let’s go outside” is antithetical to my native Texan: “Outside? Where the hot is?”
- Walking 1-2 miles daily thanks to the highly walkable life of San Francisco
- Taking up running a few years ago
- Vegetarian since January
Wherever you are, no matter your age, you can change. It may be hard, you may have to have a Harajuku Moment, but when you see yourself beat the maxim that aging is slowly falling apart, your self-esteem goes through the roof.
Getting Started With Ember 1.0 Is Easy
There has recently been some discussion about how hard or easy it is to get started with Ember. I’ve been using Ember for a while, so I don’t think that it’s that hard, personally. But there are a lot of moving pieces.
While my Ember-friend Trek is busily preparing getting started guides and the word is getting out, I have updated my Halbert software to have a tag so that you can get started right away.
WARNING: This is Ruby-biased. I’m not serving you omakase or anything like that, I just don’t know Node well enough to get something delivered quickly.
Steps
git clone https://github.com/sgharms/Halbert.gitcd Halbertgit checkout -b my-embert embert-1.0rc1bundle installrake: this starts a local web server on port:4567- Point a browser to http://localhost:4567
Screenshot

Adding a View
Here’s a gist to add a new Ember.View:
Closing
Obviously, if you want to follow along with other tutorials or start doing advanced work with the router or Ember Data this minimal harness may break down (I don’t know, I haven’t tried to break this setup ;) ), but it should let you see that creating a baseline Ember app is not impossible, or even hard!
And let me point out, further, Ember is not like YUI or MooTools or jQuery. It is not a tool that lets you do Javascript work. It is a framework for building applications in Javascript. As such, approach it and its learning curve like learning Rails. Does anyone get mad when Rails doesn’t reveal its secrets in a day or two? No, you realize that learning this paradigm will pay dividends down the road and that learning curve time is an exchange for power.
That said, get hacking. Ember is great.
Command Line Friendly Java Development Setup
I recently signed up to take an Algorithms class online via Coursera. The language of the class is the new lingua franca, Java. I do most of my development on a VPS which means I’m accustomed to doing my work on the command line with my editor being Vim. While the class doesn’t require one to use an IDE, I didn’t see that any other work flows explained how to do the assignments without an IDE.
I’ve done some work around Java build management using Maven, so I thought I would try to apply Maven’s basics so that I could do my work on my VPS. Here’s my documentation on how I did that. I hope that others in the class might benefit from this as well.
On Zero Dark Thirty
I believe in years to come, “Zero Dark Thirty” will be held in reverence with other great war movies like “Das Boot,” “The Battle of Algiers,” and “Saving Private Ryan.” All of these movies, while ostensibly about a battle or campaign were actually about something deeper: the human condition whilst under the bloody sky of war. Kathryn Bigelow’s film is special because its real task is not to visually portray events bookended by September 11, 2001 and the killing of Osama bin Laden, as the trailer or synopsis would have one believe, but rather to show a series of scenes to the audience which lead it to undergo the emotions that those who lived in that time period felt. In this regard, the film may be the best answer to “What was it like after 9/11?” Its response is not an answer but a progression of feelings and unanswerable questions.
For an example of Bigelow’s emotional chemistry experiment on the audience, the opening scene is a stunner: the screen is black and we hear a mix of recordings of emergency calls on Septermber 11, 2001. We hear the anguish of the 911 operators, the panicked (and correct) assessment from those trapped in the towers that no one was coming to help and that they were going to die. It’s a scene as scorching and destructive as the opening of “Saving Private Ryan” and Bigelow has us horrified, scared, sad, and very angry within one minute of the film’s beginning.
The next scene, however, is in a black site where the (Torture? Extreme interrogation measure?) of a detainee begins. As angry and vengeful as we were in the first scene, we are very quickly taken to wondering just how far we are willing to take visiting monstrous action upon monsters before becoming monstrous ourselves. Into this scene Bigelow inserts Maya (Jessica Chastain) who will be the audience’s proxy. Fresh-faced and squeamish (ma non troppo) in the opening scene, we’re aware that we’re going to see her hardened over the course of this movie: it’s inevitable, we know it, Bigelow knows it, and even Maya knows it. The subtlety of Chastain’s performance and keen ability to emote facially is absolutely critical to allowing her to be our proxy and, thus, to the success of the film itself. We see her sign up for a job which she knows will poison a section of her heart and kill the self she has known to that point. The scale of her sacrifice is mammoth (this topic is well-covered by Orson Scott Card in “Ender’s Game” and its sequel “Xenocide”).
Aside: I believe Bigelow could have made this movie entirely POV or just shown scenes one after another and still have achieved the goal of leading the audience on a meditation of the emotions that this arc elicits. I’m sure no studio would have gone for it given the amount of money at play, but it would have been an interesting idea.
Bigelow deftly moves us through a variety of emotional shifts: from fear to
vengeance to disgust to shame to joy to defiance to patriotism to betrayal to
sorrow to relief to bafflement in the course of this film. I can’t recall the
last film for which plot was actually so seemingly extraneous — perhaps
Terrence Malick’s “Tree of Life” (Jessica Chastain again being key to that bit
of emotional impressionism). By these measures we re-experience (for those of
us who lived through it) the emotional landscape of 2001-2012. Bigelow has
constructed an emotional stimulation device that will give future generations
the means to experience the “what was it like” of this period in history that
will compliment the facts that will be coldly written in black and white.
Hers is a magnificent achievement.
And the final scene, which is played so perfectly by Chastain, is possibly one of the most touching in film. The proxy character, when asked where she wants to go after running a seven year manhunt that culminated in lethal success cannot speak but sheds a silent tear. It’s of the elegiac tenor of “Jesus wept.” In that regard the film again accurately captures what 2013 is like. Where shall we go from here?
Ottawa Metaprogramming Presentation
Last Tuesday I had the privilege of speaking to the Ottawa Ruby user’s group via remote teleconference. While I would have preferred to be there in person so that I could interact with the attendees (despite the -23C temperature!), I really enjoyed having the chance to speak to these fine folks.
Presentation to Ottawa RUG:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | |
What Do we Mean by Metaprogramming Code Samples
Metaprogramming is definitely one of the coolest features of Ruby. I’m a
pretty “meta-” thinker, so when I saw Ruby’s capabilities around
method_missing and instance variable addition, I was completely hooked. I
hoped, by this section to expose the total “nubies” to some of that same magic.
Method Missing Example
One of the fascinating features of Ruby is the ability to not define some method when writing out the class, but rather let the “I don’t know what method that is” error get trapped by method_missing and then, based on some logic, do something other than throw the error. Here we don’t define the methods hiss or slither or strike. Rather we intercept the errors and then creatively handle them. I have no idea why I got on the topic of snakes when writing these examples.
Bonus: Show off that Ruby will let you drop parentheses. Protip: If you care about what comes back (like calling a function) use parents (as I learned from Avdi Grimm); else, feel free to leave them off.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | |
Output:
1 2 3 4 | |
Instance methods
Most programmers familiar with OO will recognize the general pattern that you make a cookie-cutter (a class) and then use it to stamp out instances of that cookies (instances). Methods are defined on the class and all the instances have the ability to call those methods.
The amazing thing about Ruby is that it will let you put methods on an instance. It stuffs those methods in the singleton class (or eigenclass or metaclass). This is where classes stuff their class methods too. When you understand how this works the whole system seems beautiful and elegant and entirely obvious. When you’re new, it seems like magic. Here’s a demonstration of the magic.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | |
Output:
1 2 3 4 5 6 | |
Notes
Here all Rattlesnakes know how to shed. The method is applied on the class, no surprise. We create a specific rattler named coily (after Q-bert). On coily we add curse. Then we demonstrate that coily can curse but texan_snake cannot. Then we put a method on the class so that all snakes know how to curse. Now texan_snake can curse and coily, who could already curse, still has his unique curse method because its definition (on the instance) takes precedence over the definition on the class. This is pretty amazing stuff.
Alternate class method definitions; the mystical eigenclass
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | |
Output:
1 2 3 | |
Notes:
Standard Ruby pedagogy will teach you about classes which can support methods that the instances will inherit. Also you will be instructed that sometimes you want to put a method on the class itself not on instances. A classic example is something like Math.sin. The sine function will never be instantiated, but it makes sense that its terminology live within a namespace (like Math). The classic way of writing this method would be def Math.sin or, while inside the Math class definition self.sin (which is clearly the same thing). The invocation that got my curiosity up was the class << self block: what’s going on there? It turns out that you’re opening up the singleton_class, the eigenclass.
These were the preliminary exposures to metaprogramming that lead me to read Paolo Perrotta’s book on the topic and to undertake my Rubyconf 2011 presentation.
http://speakerdeck.com/sgharms/practical-metaprogramming-ruby
In that presentation I lead people up a series of steps that should help them become comfortable with rolling metaprogramming into their codebase.
DSLs
The group asked about DSL construction in Ruby. I will provide a bit of context here as to why I did this. I received some feedback via Twitter on this topic and I would like to give a bit more context.
It started simply, I wanted to be able to say in an IRB like shell
@aVerb.active(TAB)indic(TAB)pres(TAB)fir(TAB)sing(TAB) and get
and get
@aVerb.active_voice_indicative_mood_present_tense_first_person_singular_number
So, for love of TAB completion, I wrote a DSL.
As it turns out, I needed to generate something like 160 unique “vectors” that every class needed to know about so that I could TAB-complete. And what if I wanted to have the order be flexible:
@aVerb.active_voice_indicative_mood_present_tense_singular_number_first_person
160 + a lot more methods to describe
And what if wanted to leave off some specifier and get a more general subset?
@aVerb.active_voice_indicative_mood_present_tense_singular_number
or
@aVerb.active_voice_indicative_mood_present_tense
160 + tons of methods to describe.
So I needed something to perform the combinatorics to generate all those vectors and to apply them onto the instance so that I could TAB-complete. So I decided to write a verb-describing DSL. The following two links show my DSL and how it is implemented. In the presentation I suggest that DSLs are best for narrow-use cases. The most easily understood DSL is Ruby programmer-to-programmer, but sometimes it is very helpful to be able to write a DSL so that business-function specific clients can influence the code without knowing Ruby. I think Rails, Cucumber and RSpec have done a pretty good job at demonstrating how well DSLs can work.
- verbvector.rb: The DSL implementation
- description.rb: The data sent to the interpreter
Conclusion
Thereafter I gave a few general comments about responsible metaprogramming:
- Always strive for clarity for other programmers to read
- Metaprogramming is not a substitute for good OO