Things I Like

Today's Collected Wisdom

2012-04-16

Confident Code

Estimates and the desire to be lied to

How Not To Run An A/B Test

Bored People Quit

Command Line Fu

and a classic American cocktail to wash it down with: Old Fashioned 101

Testing as Idiot-Proofing

2012-04-12

I was pairing on some code the other day with a coworker. It was a fairly simple thing, deep cloning an active record object and making sure the object’s collections were also cloned.

There was, along the way, a conversation that went something like:

My pair: “We should write a spec that will break if any new collections are added that don’t get cloned”

Me: “Why?”

“Because, sometimes the admin tool breaks if we forget to clone a new collection. It’s happened before a few times.”

“Shouldn’t that be part of the specs that are written before adding a new collection?”

“Well, sometimes we forget to do that”

So… we need to write a spec now, to handle adding functionality later, because we know that there’s a high probability that we won’t spec the behavior before we add it.

How many general precepts of agile development does this violate?

Well, first off it is violating YAGNI. We might never actually add another collection.

It’s also aiding and abetting a violation of TDD, which is that you would specify the future behavior before adding it.

It is also introducing a spec which doesn’t actually specify how the object behaves – it is specifying that certain implementation details don’t change.

And lastly, it is violating the idea that there should only be one reason for a test to fail. In this case the test could fail because we added a collection that needed to be cloned and wasn’t, or because we added a collection that didn’t need to be cloned, in which case our spec is now wrong. Since we don’t know the nature of our future hypothetical collection, it is probably wrong to specify it’s behavior now.

So given the situation as described, what is the right thing to test?

We’re not going to specify that all collections should be cloned, since it is already known that not all collections need to be treated the same way.

My preferred solution is to say we will write the specs for the new collection when we add it – but that does not address my pair’s concern that the team doesn’t always test drive new code.

My second favorite solution is that there should be an integration test on the admin tool that would catch any breakage.

However, I’m genuinely curious about this issue and how other people are dealing with it. Do you use your tests to prop up coding practices that are undisciplined? Or do you accept that the more you try to idiot-proof, the stronger the idiocy you encounter?

True Stories of BDD

2012-04-09

I’ve built up a healthy amount of cynicism working on the web for the last decade and a half, and it’s time to let some of it out. I just launched a little side project called “True Stories of BDD.”

It’s a place to collect and share the real story behind the “feature” you were asked to build.

I’m not sure how many people feel the same way I do, but I see a lot of stupid, evil shit happening on the web every single day. But I’m sure I don’t see all of it – so go over to http://truestoriesofbdd.com and share the ones I don’t know about yet.

Carrots and Sticks

2012-03-22

“I’m not sure if management wants to use these numbers as a carrot or a stick”

When our engineering director said that to me, in my head I heard the red alert sound from the original Star Trek. It was my WTF detector going off, again.

The numbers we were discussing were potential metrics for measuring the engineering teams performance. That’s right, once again a small management team came up with the brilliant idea of trying to measure programmer productivity. Surely, this little startup team could do what no one in the past 50 years had been able to manage. And once they had their measurements, they were going to be able to motivate their team.

Because that is what carrots and sticks are for, right?

For motivating ~~donkeys~~ programmers.

The numbers they plan to use are irrelevant. For the record, it looks like we’re leaning towards Ron Jeffries ‘running, tested features.’ This is just as likely to be gamed as any other KPI, and was originally intended as a window providing transparency, not a carrot or a stick.

The real problem is the idea that you are going to be able to move that number through some kind of reward/punishment program. Software developers just don’t work like that. Study after study has shown that programmers are intrinsically motivated to a very high degree. This means they like working, for rewards that are linked to the work itself.

Another thing about programmers is that they are creative. In Daniel Pink’s book “Drive” he talks about how external rewards actually lead to poorer performance on creative tasks. Alfie Kohn wrote that “at least two dozen studies over the last three decades have conclusively shown that people who expect to receive a reward for completing a task or for doing that task successfully simply do not perform as well as those who expect no reward at all.”[1]

So if this kind of approach to motivating a development team is all wrong – what is the right way to motivate them?

Like any good developer, I’m going to start by unpacking the requirement a little. Why would management want to motivate the development team? The simple answer is that they want to get more done. This assumes on the face of it that the team is not delivering at 100% capacity. A second less obvious assumption is that whatever the delta between capacity and output, it is related to the developer deciding not to work hard, and not the management style, working conditions, or any other factor.

The minute an intelligent team member sees you discussing carrots and sticks, the clear message they'l take away is that you think “the team is an underperforming bunch of slackers.” This is not likely to motivate them to deliver incredible work.

I think the biggest impediment to programmer motivation is the lack of an interesting problem. Paul Graham, in an essay on great hackers, talked about nasty little problems being ones where you don’t learn anything. When your day is filled with problems like that, it is like the life force is being sucked out of you. A common problem with managing developers is presenting them with a solution to implement, rather than a problem to solve.

The difference between interesting and nasty problems is often one of framing. Suppose you’re looking to boost gross revenue to look good to investors.

If you have a decent team, and you tell them “We need to increase top line revenue” then you are likely to get back a lot of questions on how you are measuring revenue, a bunch of suggestions on how to improve it, ways to identify your big-spending customers, reports on your best selling products, and potential tests for verifying whatever implementations are decided upon.

If you tell them “give out some incentive credit to all our members, and get it done by Wednesday” you’ve reduced it from a potentially interesting problem, to several nasty little problems, involving your legacy accounting and purchase systems.

Once your team has an interesting problem, the next factor affecting productivity is giving them the autonomy to solve it. Good teams need autonomy over what they do, when and where they do it, who they do it with and how they do it. Presume that your team wants to succeed, and that they want to get as much done as soon as possible. Then get out of their way and let them do it.

Of course, life is not perfect, and there will always be things that limit how much much freedom the team has. I think of this as friction in the system. When you have parts that rub together, you need to compensate by adding lubrication. For developer teams this can take several forms. My favorite is learning opportunities. Have a budget for conferences, big and small, near and far. When a team member is getting down because of the friction, send them to a conference that they want to go to. Great programmers get recharged by learning. Other kinds of recognition are nice too – take a developer to dinner, and talk to him or her. You’ll likely get some insights into what makes them tick, and the recognition of effort will not be missed by them. Personal recognition of effort is worth much more than money to a good developer (once a certain baseline compensation is met – we don’t just work for praise).

In short, treat your developers as the creative human beings that they are, and not as beasts of burden that can be motivated by carrots and sticks.

Because if they really were mules, they might wind up kicking you in the head.

[1] Harvard Business Review Sept/Oct, 1993

Update:

This post on Weaponized Metrics is another good take on the subject.

Gimme Cookie!

2012-02-13

I’m using signed permanent cookies to track users in my app, and sometimes it’s nice to be able to be able to set them in the browser manually. I’m using “Edit This Cookie” in Chrome to handle this, but it was a bit tricky to figure out how to get the encrypted value of the cookie to manually set it. Well, for future reference, I’ll put it here.

cookie_jar = ActionDispatch::Cookies::CookieJar.new(Rails.configuration.secret_token)
cookie_jar.permanent.signed[:foo] = 'bar'
cookie_jar[:foo] # => "BAhpBg==--8e5eb732858427af92a0c1eeb4f81b276a775fb8"

Using Webmock with Cucumber for Robust Integration Tests

2012-02-09

I decided that it would be a good idea to make my integration tests reflect as much of the real-world setup of my app as possible. This includes hitting 3rd party APIs over the internet. While this can be slow, it is nice to check assertions against actual responses, in case an API change breaks something.

The drawback to this was that I could no longer rely on all my cucumber features passing while I was coding on the train, or in a park, for example.

Mocking the responses would work, but I didn’t want to have 2 sets of scenarios, just for offline and online testing. I also wanted to run the same assertions regardless of where I was.

So I came up with the following pattern, which tries to use a real connection, and falls back to a stub when a connection can’t be made.

First, I use curl to record an actual response for Webmock to send. The -i option is necessary to include the headers. (The API I was using in this example accepts posts of json, and returns json responses.)

curl -iX POST -d @request.json http://www.api.example.com/do_something > do_something_response.json

Next, I set up my API stubs, in support/fake_api.rb. Note the instance variable in the After block – that a key part that we will return to.

module FakeApi
  def stub_api
    canned_response  = File.read(File.join(Rails.root, 'features', 'support', 'fixtures', 'api', 'do_something_response.json'))

    stub_request(:post, "http://www.api.example.com/do_something").to_return( canned_response )
  end
end

World(FakeApi)

After do
  @api_stubbed = false
end

Lastly, I write my cucumber step that deals with the API

When /^I do something at an external API endpoint$/ do
  api = MyApi.new
  begin
    result = api.do_something
    result.should be_something

  rescue SocketError => e
    unless @api_stubbed
      stub_api
      @api_stubbed = true
      retry
    end
    fail "Can't connect to API (real OR fake)"
  end
end

So what we do here is try the real API, and if we cannot connect, we retry the begin-rescue block using the stubbed version. We use the instance variable as a flag to prevent an infinite loop, and we use the After block to ensure that the next scenario starts with a real attempt again.

You might want to adjust this, perhaps omitting the After block cleanup to assume that if the network isn’t there for one test, it won’t be there for the duration of the run.

You could also move the flag setting into the method that does the stubbing.

My Development Setup

2012-01-30

I was working on the CSS for this blog, and reading an older post, when I realized how much my daily development setup has changed. Almost everything has changed from a couple of years ago, and I thought it might be nice to share how I get things done, since IMHO it is a pretty nice way to work.

Hardware

This has changed the least. I’m still tooling away on my 2008 MacBook Pro, although I did drop in a 500GB drive. I will say this about Apple products – they stay useful far longer than their counterparts from Acer and Dell.

Editor

Still using Textmate, although I am now starting to explore alternatives. Rubymine (which I use at work) is a total pain to work with, so right now I’m leaning towards biting the bullet and learning Vim, unless another decent editor pops up.

VCS

Github. Is there really anything that needs to be said about that?

Ruby & Rails

I’m currently using Ruby 1.9.2-p290, with RVM. My most recent project is Rails 3.1.2, but I will probably upgrade to 3.2 within the month.

Development App Server

I’m using Pow, from 37 signals, which although it has some drawbacks with regard to SSL, is still pretty nice to use locally. I got sick of tweaking Apache and Passenger config files all the time.

Testing Tools

Cucumber, RSpec 2.8 & Capybara for the basics. Fabrication as a factory for test data NullDb to ensure that my unit tests are actually unit tests. Shoulda Matchers, Email Spec, and Timecop for some miscellaneous conveniences.

Extras

I use Guard, with guard-spork, guard-rspec, guard-cucumber, guard-bundler, and guard-pow. This keeps my tests running quickly and continuously.

I’m also using TDDium to offload my CI work. There’s a post-receive hook set at Github, and they push to Heroku for me when the build is green. Pain-free continuous deployment, yay!

I’m also using Pivotal Tracker and pickler to coordinate my development priorities. With the Pivotal iPhone app, I can write stories on the train, and then pickler-pull them into my test suite to do BDD.

That’s about it – some nice tools to keep me focused on writing code instead of struggling to configure a bunch of stuff on my machine. After all, who likes shaving yaks?

Set Up A Rails 3.1 App On Heroku Cedar, Ruby 1.9, with Assets Precompiled on the Server

2012-01-07

Someone told me the other day that they had some trouble with this, so here is a quick guide to getting set up on Heroku. This approach is going to compile assets during slug compilation, which I think is the best of the available options.

Step 1

set up a new Rails app

$ rails new placepanda --database=postgresql --skip-test-unit

Step 2

put the app into a Git repo

First, I like to add the following two lines to .gitignore:

config/database.yml
.DS_Store

Heroku makes it own database connection, and I hate those .DS_Store files cluttering up my repository. Now we can get on with the commit.

$ git init
$ git add .
$ git commit -m "my brand new Rails app"

Step 3

Deploy to Heroku as soon as possible. This way we can deal with issues as they arise, not after they accumulate and become hard to track down.

$ heroku create --stack cedar
$ git push heroku master

Now we can look at the app’s status

$ heroku ps

Process  State       Command
-------  ----------  ---------------------------------
web.1    up for 10s  bundle exec rails server -p $PORT

$ heroku logs

2012-01-07T18:35:28+00:00 heroku[api]: Add-on add logging:basic by user@example.com
2012-01-07T18:35:28+00:00 heroku[api]: Release v2 created by user@example.com
2012-01-07T18:36:18+00:00 heroku[slugc]: Slug compilation started
2012-01-07T18:37:08+00:00 heroku[api]: Add-on add shared-database:5mb by user@example.com
2012-01-07T18:37:08+00:00 heroku[api]: Release v3 created by user@example.com
2012-01-07T18:37:08+00:00 heroku[api]: Config add RAILS_ENV, LANG, PATH, RACK_ENV, GEM_PATH by user@example.com
2012-01-07T18:37:08+00:00 heroku[api]: Release v4 created by user@example.com
2012-01-07T18:37:11+00:00 heroku[api]: Deploy cc64c78 by user@example.com
2012-01-07T18:37:11+00:00 heroku[api]: Release v5 created by user@example.com
2012-01-07T18:37:11+00:00 heroku[web.1]: State changed from created to starting
2012-01-07T18:37:15+00:00 heroku[web.1]: Starting process with command `bundle exec rails server -p 40738`
2012-01-07T18:37:18+00:00 heroku[slugc]: Slug compilation finished
2012-01-07T18:37:20+00:00 app[web.1]: [2012-01-07 18:37:20] INFO  WEBrick 1.3.1
2012-01-07T18:37:20+00:00 app[web.1]: [2012-01-07 18:37:20] INFO  ruby 1.9.2 (2011-07-09) [x86_64-linux]
2012-01-07T18:37:20+00:00 app[web.1]: [2012-01-07 18:37:20] INFO  WEBrick::HTTPServer#start: pid=1 port=40738
2012-01-07T18:37:21+00:00 heroku[web.1]: State changed from starting to up

Step 4

I use HAML, so I am installing the haml-rails gem, and so we can check if the assets precompilation on deploy works, I am also building out a minimal app with a HAML view, an image, and a SASS stylesheet. I’ve also installed RSpec, and Thin as the webserver at this point, because that’s just how I roll. You can see the actual commits here

Now I deploy to Heroku again.

$ git push heroku master

And it all seems to be working.

I will update this post a little later, as I add more real-world features to the app, and see if we can’t keep everything working and happy.

Step 5

O.K. – I’m back now, and installing the Compass gem and framework.

$ compass init rails . --syntax sass

edit your Gemfile to include

gem "compass", ">= 0.12.alpha.3"

in the :assets group.

Now, we’ve got to move the sass files to where Rails expects them to be, and also to rename them (to include the ‘.css’) for the assets pipeline.

$ mv app/stylesheets/screen.sass app/assets/stylesheets/screen.css.sass
$ mv app/stylesheets/print.sass app/assets/stylesheets/print.css.sass
$ mv app/stylesheets/ie.sass app/assets/stylesheets/ie.css.sass
$ rm -rf app/stylesheets

Also, we need to update the app/config/application.rb to include:

config.assets.precompile << /^((.*?)\/)?(?!_)[^\/]*$/

(The idea is that this regex prevents the pre-compilation of partials, because if that is attempted, it will fail with errors like “Undefined mixin ‘blueprint-form’”)

I usually put the above line with the regex after the line:

config.assets.enabled = true

Also, in order to use Compass properly, we need to rearrange our stylesheet folder a bit. I make a folder called

app/assets/stylesheets/application

and I put all my custom stylesheets in there.

Then I edit app/assets/stylesheets/application.css, and change from:

*= require_tree .

to:

*= require_tree ./application

This allows the compass stylesheets to render and be included separately, which I prefer. I wind up with one stylesheet for the framework, and one for the rest of my app. My layout head section looks like this:

%head
  %title Placepanda
  = stylesheet_link_tag 'screen.css', :media => 'screen, projection'
  = stylesheet_link_tag 'print.css', :media => 'print'
  /[if IE]
    = stylesheet_link_tag 'ie.css', :media => 'screen, projection'
  = stylesheet_link_tag    "application"
  = javascript_include_tag "application"
  = csrf_meta_tags

Naturally, you might want to do this all in one big file, in which case, I suggest not using the folder structure above, and instead you can just edit the application css to require the files in the right order, as the default alphabetical order is usually non-optimal. It would probably end up like:

*= require_self
* THE COMPASS STYLES
*= require screen
*
* YOUR STYLES THAT NEED TO BE ORDERED
*= require my_stylesheet
*= require my_other_stylesheet
*
* EVERYTHING ELSE IN THE STYLESHEETS FOLDER
*= require_tree .

Now we can deploy to Heroku again

$ git push heroku master

and everything should still be peachy.

Step 6

Installing blueprint.

If you’re following along with the commits on github, at this point, I’ve added my basic app functions, and I just want to style my static pages using blueprint. (The procedure will be similar for other frameworks)

First in my app directory:

$ compass install blueprint

And once again we have to move the files that get created into our assets folder and rename them to ‘some-name.css.sass’

After that, we can just deploy again, and everything is still working.

$ git push heroku master

Cleaning Up After Paperclip Cucumber Tests

2011-11-24

I love using cucumber for acceptance testing, but recently on a personal project I noticed that I was accumulating files in my public/system directory every time a paperclip step was executed. A little googling got me to a solution, which I’ve adapted below for Rails 3.1

In features/support/env.rb:

require 'paperclip'

module Paperclip::Interpolations 
  alias_method :orig_attachment, :attachment 
  def attachment(att, style) 
    "cucumber/" + orig_attachment(att, style) 
  end 
end

After do 
  `rm -rf #{"#{Rails.root}/public/system/cucumber"}`  #remove paperclip files
end

Story Anti-Patterns 1

2011-11-06

This is the first in a series, where I describe some common mistakes made when trying to run an agile team.

Anti-Pattern 1 – Visual design isn’t the whole story.

Example 1 – the team is using Pivotal Tracker as a to-do list

Typical Story

Add styling to product page.

In teams that have stories like this, there are probably a few others that look like:

Clicking on product title directs user to to product page

Product page shows detailed description and product specs

User clicks buy now button on product page

Example 2 – all the business value attached to the wrong thing for the wrong role

Typical Story

As the designer
I want a well-designed product page
In order to encourage purchases

Here, the team is trying to find the value, but they are looking in the wrong place for it. This is an example of thinking about “what do I want the user to do?” instead of “what does the user want to do?”

Forces leading to the use of this anti-pattern:

The team is using Pivotal Tracker (or their equivalent tool) to track to-do items, instead of tracking the stakeholder conversations it was meant for.
The visual designers are not working on the same team as the software developers (stovepiping).
UI design and graphic asset production is being done in a big-design-up-front fashion.
The user is lacking representation in the agile process
The development team feels pressured to put points on the board.

Problems caused by this anti-pattern:

When this pattern becomes common, you wind up with an application that spends most of its time not ready for deployment. If the demo gets rescheduled two weeks earlier, there’s a good chance you won’t be able to show many features in a complete state. Punchlists that need to be completed in order to deploy is the opposite of agile practice.

Another issue this pattern leads to is programmers waiting on designers, or vice versa. Again – not very agile. This fosters an us-vs-them mentality that is inimical to getting things done.

Lastly, when software development is iterating, but UI design and/or production is handled as a page-at-a-time waterfall, the mismatch leads to gross inefficiencies in how the look & feel gets implemented – bloated CSS, endless tweaking of styles, etc.

Solutions:

Use story templates that enforce business value consideration
Use iterative development in ui design as well as programming
Designers and developers work together on the same stories until they are done.

Using better story templates

Story templates need to show three important things:

Business value
The role that benefits from the value
The feature necessary to provide the value

The above story could become:

In order to more easily make a purchase decision
the customer
wants to see a well-designed product details page

Given an existing product named “Foo"
When the customer visits the product page for "Foo"
Then the customer should see the product details for "Foo"
And the page should be styled according to the current design specifications for the product page

Notice that there are no implementation details exposed in the story or scenario. This prevents brittle tests. The feature is written from a user’s point of view, and doesn’t concern itself with what the developers consider details, or what the designers mean by well-designed.

If what is meant by “product details” changes, this is a change in requirements that will correctly force a change in the implementation of the step. However, the feature itself does not need to change, since it is value-driven and focused on the user’s needs. Ideally, what is meant by product details should be centralized, and changed only in one place to update all the features that rely on it.

The last step could be implemented using matchers for such ui requirements as: All headlines should be blue, and all body copy should be 16px high. This then becomes a reusable step that can help catch look & feel flaws on other pages, as well.

It can be argued that tests for this sort of thing are excessive, since look and feel changes so often, and maintaining these tests creates a lot of overhead. I think that perhaps the overhead would encourage more consideration of changes, since the actual cost of doing them correctly becomes exposed. The alternative is to not care whether the look and feel is correct – if you do care, there is work to be done to ensure correctness, whether it is automated or (like most teams) manually reviewed by designers working over the shoulders of developers, nearly doubling the overhead costs.

Iterative UI design

There’s not enough space or time to go into a full discussion of agile UI development here. I wish there were more resources online, but unfortunately there is not a lot written on the topic. Briefly, though, UI development should follow many of the same principles as software development. The UI design can and should be developed in an organic, iterative fashion. Stories should be deployed as they are completed – in this system you’ll never hear “the story is done, but we can’t deploy it until the UI is finished.”

No More Us-And-Them

The UI design team should be an integral part of the development team, and should pair with software engineers to deliver stories. The pairs can work on establishing what UI elements are needed for a given story, and can add chores to create the actual assets, while using placeholders in development. Programmer-Designer pairs can go from paper wireframes to HTML prototype to working application code, together. The agile principle of YAGNI will be critical to making this system work. Design development can also benefit from DRY thinking.