Guerrilla user testing in an agile world - Part 1

October 16, 2014

A bit of a warning. When I started writing this post I had no intention of doing a 2 part post. It was supposed to be a quick introduction to a session I will be presenting at SharePoint TechFest here in Houston on November 5th but there was just so much information that I felt needed to be presented and it just continually grew. So unfortunately this is will be a 2 part piece.

  • Part 1, the one before you, will focus on the importance of why you should be asking your users as early and as often as you can during the process of developing your next project.
  • In Part 2, I will focus on how to get user feedback by providing some quick and effective methods for getting feedback from your users that will prove invaluable and quite possibly save your next project from falling short, or worse yet, never getting adopted by your users.

So with that out of the way, I give you Part 1 - Why many projects fail or worse yet never get adopted.

The cost of project failure is immense. Estimates done in 2009 put the cost of IT project failure in the $6.2 trillion, yes that’s a trillion with a ’T’. According to the study done by Roger Seasons, nearly 25% of all IT projects result in failure. In 2012, Michael Krigsman challenged these numbers and asked Gene Kim and Mike Orzen to take a second look into the numbers and make a more accurate estimate of the impact of IT failure on the economy.  

Kim and Orzen took a conservative approach and applied a 20% failure rate to start their calculations. 

For just the Standard & Poor 500 companies, aggregate 2012 revenue is estimated to be $10 trillion. If 5 percent of aggregate revenue is spent on IT, and conservatively, 20 percent of that spending creates no value for the end customer - that is $100 billion of waste!

 When looking at where IT money is spent.

If we conservatively estimate that 50 percent of global IT spend is on "operate/maintain" activities, and that at least 35 percent of that work is urgent, unplanned work or rework, that's $980 billion worldwide of waste!

Their conclusions.

What reward can we expect through better management, operational excellence and governance of IT? If we halve the amount of waste, and instead convert it into 5x of value, that would be (50 percent * $1.2 trillion waste * 5x). That's $3 trillion of potential value that we're letting slip through our fingers!

Kim and Orzen have clearly done some legwork and come to what appear to be a bit more realistic numbers. In either case, though it’s not exactly clear what the criteria for ‘failure’ was. 

I won't dispute that “better management and governance of IT” are not significant contributors to a successful product. That being said, a project that is managed by the greatest of managers, running a project so flawlessly that a swiss watch maker would envy, is still open to complete and utter failure. What’s worse it is a chance at failure that can be all but eliminated with a much more simple solution: Ask the users.

 Yes. I said ask. Ask users. Ask users as early as possible. Ask early, and ask often.

Some of the typical responses, or reasons offered for not doing user testing or research include: 

  • It’s too hard
  • Don’t have enough money
  • Don’t have access to users
  • Don’t have the time
  • We are the users
  • We already know what the user wants

Now, before going any further I am sure after reading this list, one of the first things that popped into your head was the often overused and more importantly misunderstood quotes from what has been attributed to Henry Ford.

If I had asked people what they wanted, they would have said faster horses.
- Henry Ford

There are a couple of issues with this. First and perhaps least important, Henry Ford never said it. Certainly none of his biographers attribute it to him, and nobody actually has any record of it being said. However, the second issue is significantly more problematic. This quote is repeated ad nauseum to justify dismissing user feedback and participation. Our misappropriation of this quote just further reinforces this. Even if Henry Ford had said it, he wasn’t looking to solve the “Horse” problem. Henry Ford was trying to fix the mass production problem, how to get more automobiles cranked out. 

No one said they wanted faster horses, they wanted less horseshit. -Erik Flowers

 Minimal effort will reeal, that Horses were in fact an issue in the late 1800’s and early 1900’s. However, their slowness was not on the list of issues anyone had. In fact, there were more horse related traffic deaths per capita than automobiles today. So asking for ‘faster horses’ would be highly unlikely.

So with that little hurdle cleared I return you to our regularly scheduled post:Asking your users. 

 Think about it. If you knew people were not going to use your website or app, or aren’t going to like it, wouldn’t you want to know before you put all the effort and resources into building and deploying it? 

Sure, your employees are somewhat a ‘captive’ audience when it comes to your company intranet, or software but how many times have you heard “happy employees are productive employees.”? You absolutely have to weigh user wants, goals and desires against business objectives but the more you can merge the two, the more your project is assured of success.

 Next month: Guerrilla User Testing in an agile world - Part 2. In that post, I will focus on how you can ask for user feedback via some quick and effective methods.

And if you’re in the Houston area, think about registering for the upcoming SharePoint TechFest Houston event at the NRG Park.

Resources:
Worldwide cost of IT Failure (revisited): $3 trillion - zdNet April 2012
Worldwide cost of IT Failure: $6.2 trillion - zdNet December 2009
Critique: $6.2 trillion Global IT failure stats - zdNet December 2009
Erik Flowers - "No one said they wanted faster horses, they wanted less horse shit."

Information and material in our blog posts are provided "as is" with no warranties either expressed or implied. Each post is an individual expression of our Sparkies. Should you identify any such content that is harmful, malicious, sensitive or unnecessary, please contact marketing@sparkhound.com.

Meet Sparkhound

Review our capabilities and services, meet the leadership team, see our valued partnerships, and read about the hardware we've earned.

Engage with us

Get in touch with any of our offices, or check out our open career positions and consider joining Sparkhound's dynamic team.