Confessions of a reformed project estimator

A lot of people know me as the “estimates guy” thanks to my conference sessions on estimating Drupal and web projects. But in the years since I’ve had reason to revise my opinion on the use of estimates. These are my realizations and insights since.

Some five years ago, I made the decision to try and overcome some of my fears. One of them had to do with presenting. Perhaps “fear” wouldn’t be the right word, discomfort, is a more fitting description. It was something I wanted to address as I thought presenting could be a lot of fun if I could just figure out how to feel comfortable on the stage. This led to me to make several presentations on estimating web projects. The reasons I chose estimation had to do with our company and where we were at the time.

It all began in the Old Town

Back then I was working at NodeOne, the company I co-founded back in 2008. We were a young, energetic web development and design agency who’d specialized in Drupal, a very popular open source content management system (CMS). And we did it at exactly the right time. From our hip and impressive but poorly ventilated offices in Stockholm’s Old Town, we churned out several high profile websites. It was a fun time and offered a lot of opportunities for growth and learning. Drupal is extremely flexible and adaptable which meant we could build almost any kind of website, ranging from small business sites to complex community sites with internal logic.

One of the problems we had to tackle early on had to do with figuring out how much to quote a customer. As you may be aware of, most companies and organizations procure website development services through an RFP-like (request for proposals) process. First, someone higher up makes the budgetary decision to procure a website. Next, internally or with the use of a specialized consultant, a list of requirements is drafted and approved. The requirements, along with the terms are sent out to potential bidders. Bidders are expected to return with a plan for performing the project, a document describing why they’re the best or most suitable and a price as well as prices for add-on and follow-up services. This comprises a tender. In legal terms, a kind of option that commits you to sell the tendered services at the given price and under the terms stipulated.

Most of these documents are boilerplate. You can prepare templates where you insert the customer’s name and the seller’s name to customize them. But there was one part that took more work effort than just running search and replace in a word processor, it was the price.

The way we worked, price was calculated a bit like this:

Estimated number of developer hours × hourly rate development
Estimated number of designer hours × hourly rate design
Sum of the above × 0.2 × project manager rate
Estimated fixed price items such as web hosting set up
Sum of the above × Applicable volume or other discount based on quote size, number of hours or type of customer (for-profit vs non-profit)

Here’s an example using fictional rates in euros:

Developer hours: 500 × €85
Designer hours: 200 × €95
Project manager hours: 140 × €115
Website set up, one time fee: €500
5% volume discount for projects over 500 hours: 78,100 × 0,95
Total price: €74,195

I believe most web development shops used similar formulas to derive their prices. So there was nothing strange about this approach. It seemed really obvious. We had no reason to question it and it won us enough work stay liquid.

As you can see, the key numbers here are the estimated hours for design and development time. We defined these types of work differently and we also argued that design was more strategic and inherently more valuable to the customer, why we charged more for it. I may not agree with that now but that’s how it was done back then.

The tricky part was coming up with those time estimates. In other words, figure out how much of our teams (or “resources”) would be used in order to complete the project for the customer.

Our use of the time estimate sheet

My NodeOne co-founder Thomas Barregren suggested an old proven method of using a spreadsheet to calculate the estimates, but with the added factor of using a risk estimate in addition to a number. In this context, risk refers to the possibility that the estimate is wrong and the actual outcome exceeds the estimate.

Normally, you’d line up the things you expect that needs to be done like so and estimate each in hours:

Item                                   Estimate
Create blog content type                      1
Create blog view listing                      2
Create page content type                      1
Set up captcha module for comment posting     2

But adding a risk factor allows you to weigh in the fact that some items are potentially more complex than they might seem and as a result take longer than we thought:

Item                                   Estimate      Risk
Create blog content type                      1         1
Create blog view listing                      2         1
Create page content type                      1         1
Set up captcha module for comment posting     2         2
Integrate with Facebook API                  10         3

To make things simple, you can just multiply the two. So the estimate for the Facebook API integration would be 30 hours, and not 10 (which is a very optimistic number).

He also recommended the use of an additive sequence of numbers, such as a Fibonacci sequence, to increase estimation accuracy. Wikipedia defines a Fibonacci sequence like so:

“By definition, the first two numbers in the Fibonacci sequence are either 1 and 1, or 0 and 1, depending on the chosen starting point of the sequence, and each subsequent number is the sum of the previous two.”

So the first numbers in the Fibonacci sequence are:

1, 1, 2, 3, 5, 8, 13, 21

We simplified it and used this sequence instead, always rounding up if uncertain:

1, 2, 5, 10, 20 and 50

The power of using a sequence derived using the above formula, or something similar, is that it reflects the fact that the potential error of an estimate stands in direct relation to the size of the estimate.

By using a fixed sequence and risk factors, we suddenly had a tool for creating estimates in a consistent manner. Consistency is paramount in a business as consistency means predictability. We needed to be able to know how much we were going to be able to bill in the next few months and that required that projects didn’t run over budget and hopefully even left a little room for additional profit.

The job of estimating RFP’s fell on the shoulders of a few of us. Mostly the most experienced developers who’d seen enough projects to know what problems and challenges could lie hidden in the murky depths of innocently looking requirement items such as “Facebook integration”.

I ended up doing a lot of estimating. I was the guy who said “hold on, this looks fishy” and recommended taking a defensive stance. Many of our projects were running over budget and the company’s profitability already took hits. The challenge was to present an attractive bid to the customer while at the same time remain profitable. All digital agencies that rely on bids and hourly billing face this challenge and there are some creative ways to address it.

As I began working with this technique and our estimating tool, i.e. the spreadsheet I started developing techniques and expanding on the ideas or risk and using semi-additive sequences to manage risk and increase accuracy. These ideas could fill a blog post of its own. They were things I derived from books I read. I realized why estimates usually were so far off and I proposed ways to make them more accurate by relating requirements to previous work and whether you were familiar with the risks inherit in estimating something you’ve never done before.

These ideas and insights eventually led me to create, design and hold several presentations about the techniques I’d developed for our use. Presenting them to others forced me to find a way to communicate my workflow into steps that could be followed by anyone with sufficient experience as a developer.

It took 80 hours to make the first presentation but I managed to explain and describe them in a way that made it possible to learn them in 40 minutes or less. Even to this day, people approach me at conferences and tell me that they’re using the spreadsheets I’ve designed and that they’re getting high precision estimates out of it. I’m always happy to hear that something I developed for my own use has been valuable to so many others. Back in 2011, I was even interviewed by Lullabot about it at DrupalCon Chicago for their Drupal Voices podcast about the estimation techniques I was presenting.

In retrospect, I believe many of the opinions I had back then in 2011 have changed. I have, since then, changed my thinking about the value and use of estimates. Which is the topic of this blog post.

So what’s the problem?

All this sounds nice and well, useful. Yes, it absolutely is. Estimates are a core competence in any successful web development firm.

But over the year since 2010 when I first started talking about these things at DrupalCon Copenhagen, I’ve changed my view of the role of IT projects and how we, as consultants, should price our services.

I would go so far as to say that I’m critical towards a way of working that I was one promoting through talks and presentations. A way of working that depended on using estimates for the wrong things.

Back at NodeOne, like most web agencies, we applied so called cost-plus pricing. In cost-plus pricing you sum up the expenses or costs for delivering a good or service and add a fixed markup. All our prices stood in direct relation to our costs, not in the value we created for our customers. This was a direct consequence of using hourly billing, time sheets and time estimates for budgeting.

We eventually switched to non-fixed bids and offered customers the choice between a running bill or a fixed one but with an extra 25% markup to cover unexpected costs. As I recall, most customers preferred the running bill. At the same time, we also stopped planning projects in detail in advance and took an agile approach to project planning. This meant variable scope, fixed budget and fixed deadline. But the scope and backlog was still measured and billed in hours in classical fashion. We just didn’t have to know how long each specific item would take right at the beginning, allowing for more flexible planning and for the customer to change their minds during the project itself.

Unfortunately, even with all the great advantages of Scrum and agile, cost-plus pricing and hourly billing have a number of problematic consequences.

Puts a cap on profits

It creates a profit cap, consisting of our markup, on how much money we could earn since projects almost never took less time than estimated. Many successful web development shops make accurate estimates and hover at around 10-20% profitability but those are exceptions. In variable scope agile projects, the full transparency and time logging of backlog items means that you generally get your time covered bill-wise, but your profit will never exceed the cap.

If you think profits are somehow associated with “bad capitalism“ I think you need to widen your perceptions a bit. This isn’t about taking people’s hard-earned money but about creating a financially stable and strong business, where people never have to worry about their jobs but can focus on doing a fantastic job.

Links price to costs

It links price to costs. Two concepts that have very little to do with each other. Imagine a bottle of water, how much would you pay for it in the super market compared to in the desert? As this example shows, price is contextual while estimates strive to be consistent in order to allow predictability. Effective pricing takes context into account.

Leads to commoditization

It opens the door to a form of of comparability that enabled commoditization of what we were doing. A lot of companies shop development services based on hourly price, forgetting that this is a highly specialized service and hourly price doesn’t reflect the total price of a project. The way to stay competitive is remain hard to compare with others.

Billability, not customer satisfaction, becomes the key measure

It creates a culture where the focus is on billable time, not delivered or perceived value. The company’s internal KPI’s focused on resource allocation and billability, not indicators such customer satisfaction such as NPS (net promoter score). As a result, there’s a high risk that customer satisfaction suffers.

Enforces a factory production line kind of workplace

It made it impossible to introduce a truly flexible workplace such as ROWE where fixed working hours are replaced with results, goal-setting and mutual trust. A form of working which has been shown create outstanding employee satisfaction and motivates and attracts the kind of people you’d like working for you: independent, driven and trustworthy individuals.

Creates the circumstances needed for some really bad customer relationship mojo

It set the stage for seller—buyer conflicts where the seller and the buyer would be in strong disagreement over the scope and the requirements. The buyer wanting to maximize what they felt they were getting and the seller wanting to minimize time (cost) spent. In the end, arguing over something that may not be all that relevant. This doesn’t necessarily happen but cost-plus pricing derived from estimates make it more likely.

Makes it harder to design and build for impact

It hinders the adoption and use of outcome-based frameworks such as impact mapping and impact and effect management. These frameworks focus on the desired benefits or impact of what you’re building, putting users first and deliverables second. I’m not saying it couldn’t be done. Only that it’s harder when what you focus on are deliverables, pieces of code, which themselves aren’t valuable but become valuable when they’re being used by someone.

Makes adoption of agile project methods harder

It makes it hard to truly embrace agile since all estimates were hourly, even if we decided to call them “story points” in agile fashion. In the end, the project has a budget of N number of hours and every item in the backlog will take some hours out of that, even if it’s estimated to X story points. In the end, the product owner needs to track the time and staff needs to report the time. It’s inevitable.

So, yep, a lot of really bad stuff. Enough to fill a blog post of its own.

As you can see, estimates are great for estimating costs but they should never ever be used for pricing. Keep estimating for reasons of planning and knowing where you break-even, but don’t let the price discussion be influenced by the estimate.

But is there an alternative to estimates for determining price?

Yes, there is. It’s called value-pricing and it is a more honest, effective and mutually beneficial way of working. But it takes work and it cannot be calculated using a formula. It requires you to develop a marketer’s instinct for what drives your customer to buy and care about their return on investment, how they perceive you and your customer service. It’s worth the effort though as value-pricing creates stronger customer relationships and increases the value you provide to your customers.

More to come

I’ll be writing a lot more about value-pricing for web and digital agencies in the coming weeks.

If this seems interesting and something you’d like to learn more about, check out my startup's blog: The Bondsai Blog

—

Photo by David Mollière: https://www.flickr.com/photos/davidmolliere/4925490525/in/photolist-