grep needle haystack |grep special_needle

or his inverted cousin:

grep needle haystack |grep -v unspecial_needle

These are common staples of searching through text files.

You should stop using them. Now.

You can do much better by writing more consice regular expressions and using ack or one of its relatives (ack-grep, or rak).

The primary virtue of these commands is that they use the Perl regular expression engine. Most programmers with experience in any of the major scripting languages will find this more comfortable than grep’s use of the GNU regex syntax.

I recently encountered a need to search through many files based on a complex regular expression that required lookahead and lookbehind asserions. I have no idea how that would work in GNU regex land where, honestly, I still have a hard time getting simple capture and alternation. After learning look around syntax, I was glad to know that ack could directly implement it.

Given haystack, a text file:

tin needle
silver needle
lead needle
ocelot
monkey

Find the needles (this is where most grep users get to and never leave):

$ ack needle haystack
tin needle
silver needle
lead needle

Look at that one character shorter than grep and just as easy. Now if you want the silver needle, the unsophisticated, greppy way of doing this would be:

$ grep needle haystack|grep silver
silver needle

This sucks. Try:

$ ack '(?=silver).*needle' haystack
silver needle

Or, “all things in the haystack that are needles, but not the lead one”

$ ack '^(?!^lead).*needle.*$' haystack
tin needle
silver needle

I know there’s a lot more to the power of the lookaround assertion, but if I can re-train myself out of this habit I think it’ll be a big win. Granted, for smaller searches the “double-grep” method is probably fine, but any time you’re doing a recursive descent and are looking for true needles in the haystack, ack’s is the superior approach.

The case that I was working on was where I needed to search all my Rails models for all named scopes that did not use the lambda form and I wanted five lines of context around the matches so that I could be understand the behavior.

ack -C5 'scope(?!.*lambda)' app/models

That’s gold worth searching for.

I love the idea of these little machines lurking about hte house doing all sorts of automated stuff for us behind our back. While it’s clearly underpowered to do advanced photo editing, most of our requirements for household computing (yet) are fairly low power with frequent gap time.

The Vim editor has a built-in function for preserving your editor state — which files you had open, which tabs they were in, etc. — called “Sessions.” In my experience, few Vim users, even experienced / advanced users, take advantage of this feature and that is a shame. Sessions allow you to get you back into the working context of a given problem immediately . If you’ve avoided a needed reboot for software update or wished you could save the context of your editing as related to a specific problem, you should take the time to learn a little bit about sessions.

Curating an Editor State

As developers, we generally start with a single file (“QA says there’s a bug in this view”) and then realize it has a relationship with another file (“Hm, but I need this as the data source / dependency / etc.”). Over time we come to recognize that there are several files whose relationship and interaction creates a “feature” we want to work on. I call this collection of files that represents the means to the solution the “curated list.” We groom our curated list into a user interface that helps us address the problem:

The curated list for this blog post

Finding yourself needing to reboot, where you would wipe away this solution context like some Tibetan sand mandala might put you into a panic routine:

“Which files do I have open? Which directories am I in, which server processes are running?”

The Oatmeal comic hits home:

To save your state all you need to do is run :mksession. By default this will create a file called Session.vim in your working directory. Its contents are a full list of Vimscript commands that will be issued when you launch Vim with a reference to that file with vim -S session_file_name at a later date. Reading this file is also a great way to learn some powerful Vimscript commands.

The commands it requires to re-build your Vim session as it is right now

Standard Vim file-write semantics still apply To overwrite the old session it’s :mksession! and :mksession session-from-after-lunch.vim It is this latter usage which allows you to create ‘topic based sessions.’

Topic-Based Vim Sessions

The average Vim session is very small (kilobytes) and they can be made cheaply (less than a second) therefore, as I work on problems I’ve taken to creating multiple Vim sessions so that I can address a feature and come back to it painlessly.

Presently I’m experimenting on creating sessions based on bug-tracker stories. The session files (ending in .vim) are included in my global .gitignore file. Given that a certain feature may be revisited at any point, it seems a good idea to keep each in its own session so that I can pick it back up again for subsequent bugs / improvements. AND should a refactor add a new file or drop one, all that’s required is a simple mkession! to write a new session.

Gotchas

By default mksession saves blank, buffers, curdir, folds, help, options, tabpages, winsize. Depending on how heavily customized your Vim configuration is, you may find these session commands clashing with your default options. For a heavily customized Vim installation (like Janus Vim) this can create some erratic results. To prevent any clashes, I have modified the default sessionoptions configuration in my .vimrc to be the following:

” Sessions in Janus are overwrought set sessionoptions=blank,buffers,curdir,folds,help,tabpages,winsize

Reference

As sessionoptions seems to suggest, there are a wide variety of things that can be preserved. :help sessionoptions and :help mksession in the on-line Vim help guide should help you tweak your configuration to perfection.

I can even seen an argument for adding a :mksession! as an autocommand, so that your session is written on every file-write or buffer-change event. I don’t (yet) see a need for that, but it’s great to know that Vim has that capability.

Conclusion

With OSX Lion now preserving open application state, browswers now allowing you to save tab state, and Vim now allowing you to preserve buffer state, you can live in less fear of a power outage, an accidental reboot, or a gray screen of death.

set editor="mvim --remote-tab-wait-silent "

This is pretty neat. When I go to reply or compose, I’m thrown into a new tab in MacVim and I can compose / reply there. After hitting ZZ or :wq!, mutt properly handled the signal, but didn’t change OSX’s focus back to the terminal window where Mutt was running. By changing the editor to the following configuration:

set editor="osascript $HOME/bin/mutt_edit.sh"

I was able to tell Mutt to send my temp file (for the edit) to osascript, a utility that runs Applscript. In mutt_edit.sh I provided a bit of a wrapper around the mvim command such that it told OSX to re-activate my terminal session after the editing activity finished. Here’s the code:

on run argv
  tell application "Finder"
    do shell script "mvim --remote-tab-wait-silent " & item 1 of argv
  end tell
  tell application "iTerm (1.0.0.20111020)"
    activate
  end tell
end run

And that’s all it took.

Typical scene of getting laptops set up (source)

In these activities I have encountered a powerful enemy to successful teaching that we, as organizers and teachers, should remain vigilant against: negative confirmation bias.

Negative confirmation bias is when a student walks into a situation expecting that s/he is not going to succeed and then proceeds to look for data to back up that negative view. Why students have this negative view is a subject for another post, but it is my belief that it is at the heart of why there is such poor diversity in high tech.^* At development workshops / boot camps, students encounter a dangerous confirmant of their negative bias in one the earliest steps: “Bootstrap the setup using git.”

Facilitators must take an active role in reminding students that the installation / bootstrap step bears no inherent reflection upon their ability to enjoy, learn from, and perform well in the class.

It’s surprising since planners and facilitators work so hard to make so many other aspects of the student’s engagement easy so as to counter the possibility of negative confirmation. Specifically, Railsbridge sessions do a great job:

1. Delivering an energetic environment that checks: “I knew developers were boring, I don’t belong there.”
2. Saying “welcome” upon check-in that mitigates: “Everyone ignored me, I knew I didn’t belong there.”
3. Pre-communicating the transport situation that checks: Couldn’t park, probably didn’t belong there anyway.”
4. Building “pods” where informal collaboration occurs that checks: “*No one saw I was confused, I knew everyone there was smarter than me. I knew I didn’t belong there. *”
5. Providing power strips, snacks, job prospects, etc.

But then we run the risk of glossing over something that seems very simple. We command:

“git clone a respository so that we can all start on the same page.”

This is a simple operation for anyone who has it as part of their daily workflow — something that we do with the casualness of deleting a file. But it is easy forget that this culminating activity is built upon a series of installations and configurations. To wit:

• “Git clone a respository so that we can all start on the same page.”
  • git clone http://github.com/some/repo railsbr
  • get git
    • get [homebrew](http://mxcl.github.com/homebrew/)
    • get gcc and its libraries
    • get gcc from the Kenneth Reitz bundle OR
    • get gcc from XCode for Lion OR
    • get gcc from a legacy XCode
  • Tacitly understand the concept of a SCM
  • etc.

The challenges, tweaks, and reliance on facilitators’ judgment that “Oh don’t worry about this, it’s not fatal” or “Oh, right, we need to copy XCode from the USB key” can easily upset the student’s steady growth in confidence at the most critical, incipient point. Facilitators would be wise to handle this particular snag carefully.

The last session I participated in focused on essential web development skills: HTML, CSS, Javascript, and jQuery. git has absolutely no direct relationship to style of development.^** Yet as we stepped through this stack of activities, I could almost hear the biases finding confirmation: “If I can’t get this working and the thing hasn’t even started yet, there’s no way I’ll figure out the rest.”

BUT once the students got past the installation snarl, they were well on their way and, from the feedback, it seems all the students (and teachers!) learned a lot and had a great time. It is absolutely essential that facilitators recognize this weakness in our curriculum and set expectations around it properly.

Big Nerd Ranch founder Aaron Hillegass has a great section in the beginning of his Cocoa programming for Mac OSX. I learned a lot from Aaron several years ago and I think of this story whenever I get disappointed or let down in my battles learning anything. Heck I’ve even repeated a paraphrase of it to some of my students at events like this. Here is is from the source:

“I used to have a boss named Rock. Rock had earned a degree in astrophysics from Cal Tech and had never had a job in which he used his knowledge of the heavens. Once I asked him whether he ever regretted getting the degree. “Actually, my degree in astrophysics has proved to be very valuable,” he said. “Some things in this world are just hard. When I am struggling with something, I sometimes think ‘Damn, this is hard for me. I wonder if I am stupid,’ and then I remember that I have a degree in astrophysics from Cal Tech; I must not be stupid.”

Recommendations for Organizers

If organizers use git or other bits of hacker-friendly technology to get your students bootstrapped, it is imperative that you explain that success with git has no inherent bearing on success with web development in general, ruby, or rails in particular.

It’s also worth bearing note that this applies to many of the other macroscopic activities that happen at workshops. rails g migration renameUserToClient runs on a similar “stack” of assumptions required to get the framework. The student will likely lack the familiarity required to discern the difference and conclude that:

difficulty bootstrapping a tool == difficulty using the tool == I am stupid

Here are some approaches that I’ve tried. Obviously, no approach is perfect for all students, but here are some ideas.

• “This is magic, this is weird stuff and it’s something you’ll learn, but for today, let’s just follow the instructions and not worry about the particulars. It’s not programming, it’s just a tool.”
• “The goal for us, as teachers, is to get everyone on the same page. This is just administration, not really coding $TECHNOLOGY_NAME so much. But we need you to help us get your machine ready and we’ve chosen one of the most difficult ways possible to do it (ha-ha).”
• Use the “stack” metaphor above to explain that all of these little steps add up to completing the one simple task of getting the student on the “same page” by cloning a repository
• Explain only as much as the student asks for. For example: “What’s git?” It’s a way for synchornizing files from a master source. We want all of the students to have a synchronized “base” state. NOT: It’s a DCVS that uses linked lists, has lightweight branching, and no designated HEAD.

The hard part is to become interested, to wake up early, to find parking / take transit, and to have the courage to actually open the door and say “I don’t know, but I want to know.” A bunch of noise about libraries should not halt that process or that progress.

Footnotes:

* Approximately 28% of high tech work is female; 2% in open source.

** In Railsbridge’s other sessions on Rails the linking is more obvious as one must use git to deploy the code to public servers. But acumen with an SCM certainly has no necessary correlation to success in the field of software development.

Here’s how this post was made:

1. :Blog np created a new post
2. Moved down to the Categs line and then: :Blog cl
3. /Techno to match the right line. Hit enter.
4. Vimblog just copied the category name to the yank clipboard for me and then…
5. p to paste it in
6. G to the end of the file
7. o to start typing
8. Typed Daniel Miessler
9. v2b\gifl did a Google I’m feeling Lucky to his site and wrapped it in a markdown link
10. More editing
11. :Blog draft (because it’s still alpha-grade code) so I could check it out in WordPress
12. Took a screen-shot of vim to put in the post
13. :Blog um ~/Desk/vimb
14. Vimblog pasted the url link to the image that makes this work:
15. Published this post with :Blog publish
16. Damn it feels good to be a gangsta

I had to admit, this was a pretty compelling feature provided by Textmate’s “Hyperlink Helper” bundle. Its lack definitely hindered my drive to blog in vim.

To that end, I have created my first Vim plugin and port this sexy capability to Vim.

The port is called GIFL which is not some sort of obscure term for attractive septugenarians, but rather is Google I’m Feeling Lucky.

With this installed you can do things like:

Imagine you have the following text, with your cursor at ‘*’:

*lindsay lohan

Now enter:

\gifl2e

You now have:

[lindsay lohan](http://www.myspace.com/lindsaylohan*)

That is, the Markdown code for a URL. That’s pretty handy. This loads up in the global namespace, so you can use it anywhere and everywhere. I’ve found it suprpisingly handy in places other than a blogging context. It’s still very, very, very alpha, but it gets the job done.

I must say that coding in vimscript is a bit of a challenge, but I have been greatly helped by Steve Losh’s Learn Vimscript the Hard Way.

By the way, thanks to Pedro MG’s vimblog project, I was able to write this post in Vim, with GIFL doing nearly all the linking.

I think that both GIFL and Vimblog need to be polished a bit before I can say that, together, they make it as easy as it was in Textmate to be a blogger, but it’s definitely closer. Thanks to Macvim + Janus as platform, this is close.

“Practical Metaprogramming” hosted by Confreaks

• Number of days when pairing occurred: 5
• Number of hours spent pairing: ~37
• Number of stand-ups: 5
• Time spent in stand ups (week): ~ 50 minutes
• Brown Bag Lunch: 90 minutes
• Emails received within project team: 3
• Emails received from client: 1
• Skills Worked On: HTML5, CSS3, Rails, Backbone.JS
• Times astounded by my pair’s refactoring skills: 2
• Times astounded by my peer teams’ approaches to managing complexity elegantly: 2

Today on my first day at Carbon Five, I got registered for my accounts: email, tracker, etc. And of course:

I USE HIPCHAT, Y U NO GUY.

Hipchat’s founder, Pete, tells the story of Hipchat aweseomeness here.