The HWYF Test

“How would you feel if you could no longer use this product?”


I’ve done a lot of work with NPS – both using it as the “one metric to rule them all”, as well as  using NPS ‘verbatim’ feedback to inform the Confluence product roadmap while at Atlassian.

But one of the questions I asked customers was: “What would you use if you didn’t have Confluence?” The answer was always: “Nothing! There’s nothing else that can do this!”

I love NPS, and it’s a great way to gauge if one company/product is ‘better’ than another if you’re looking to make an angel/VC investment.   But a low NPS score (“I won’t recommend…”) might not be a good indicator of stickiness.

That’s where the HWYF test comes in! Ask the simple question:

How would you feel if you could no longer use <product>?

  1. Very disappointed
  2. Somewhat disappointed
  3. Not disappointed
  4. N/A – I no longer use <product>

Only count the percentage of people who answered 1. Very disappointed. According to Sean Ellis, who first wrote about the HWYF test, you want that number to be more than 40%. If it’s significantly less than 40%, you haven’t achieved product/market fit.

So what do you do? One thing is to start segmenting. Is there a difference between male and female users? Do software engineers love your product, but business people don’t? Either double down on the market where you do have product/market fit, or consider rethinking your product.

Credit: Sean Ellis via Trevor Owens in The Lean Enterprise


I did two separate HWYF tests on Twitter, to determine the HWYF score for iPhone and Android phones, both of which I’d argue have achieved great product/market fit. Unsurprisingly, the HWYF score in both cases is well over 40% (but still under 50%)

iOS vs Android – a Response to @gregmuender

@gregmuender lasted 37 days hours on Android and wrote about it here.

He makes two valid points: 1. Android is more of a mission to set up and 2. iPhones come with better charger cables. 

I agree with Greg that it would be nice to have things that “just work” out of the box. And Apple has played perfectly to the fact that the vast majority of people on the planet, when it comes to having to _learn_ anything tech, are muppets. 

But what if you’re not a muppet? What if you speak 4 languages and want a keyboard that automatically figures out, and actively learns, the language you’re texting in (I’m talking about SwiftKey)? It’s actually a feature that’s truly useful, even without dancing ninjas.

So that when you type in “moet” (Afrikaans for “must”) it doesn’t auto-correct it to “Moët”.

Every. Single. Time. 

What if you want to be able to simply drag & drop your MP3s to a folder on your phone and they will simply be found and be played? Instead of having to first do a huge iTunes import mission. 

Same goes for videos! Why the hell must I struggle converting them to a special Apple format first?

Then there’s the “Photos” app in iOS 7. I have ~800 photos in the “Camera Roll”. Many of them are 5 pictures of the same thing – parents of kids know what I’m talking about. So I pick the best one and add it to an Album (a mission unto itself). I keep doing this until I’ve taken the 100 photos I want to keep and put them into albums. So now I Select All in Camera Roll and select Delete.

*BOOM* All the photos you had meticulously copied into Albums are now also deleted! 

Yes, you get a warning… so you get a chance to, erm, … yes… try to figure out which  100 pictures you actually put in albums! So that you can be sure NOT to select those ones. (No, my dear Android users with no exposure to iOS, I’m not kidding. That’s really how the Photos app works.) 

Is iOS “easy”? Yes, maybe. Does it make anyone more productive? 

No way.

WordPress: Please Don’t Make Me Think

About a month ago I moved my blog from Posterous to a self-hosted WordPress installation. I didn’t bother setting up the Akismet comment spam filter at the time and started noticing quite a few spam comments coming through. From having previously worked with WordPress, I knew that I needed an API key. 

These days, it’s a bit easier to get, as you can simply sign up for a account which gives you a whole host of benefits when using a standard installation on your own server. 

I clicked on Plugins and selected “Settings” under Akismet. I entered my API key and started frantically looking for a “Save” or a “Save Settings” button. I found none, and thought that maybe it would auto-save once I navigated away from the page – a trend in desktop software, especially since Mac OS X’s System Preferences started doing this. 

I noticed a button saying “Update Options >>” (see below). I clicked on it to see what the “Update Options” might be, expecting a popup where I could choose whether Akismet would update automatically or manually.  The little double arrow indicated that further options would appear. 

Screen Shot 2013-05-10 at 9.46.03 AM

Ha! I was wrong. A message appeared saying; “Options saved.” 
Screen Shot 2013-05-10 at 10.02.38 AM
So why doesn’t the button say “Save options”? 
Steve Krugman writes in “Don’t Make Me Think” about naming buttons. In particular, regarding a Search box/button: 
It’s a simple formula: a box, a button, and the word ‘Search’. Don’t make it hard for them – stick to the formula.” 

Skyrove Customer Feedback – Some good, some bad

The good (from someone who had a complaint that we were able to resolve):

“Thank you for this offer. I am always amazed by the service provided
by Skyrove. It is so completely different to any other company in
South Africa. Whenever I ask anything, I get a hundred times more,
than I originally ask.”

The bad (from someone who mistyped the voucher numbers and didn’t like
it when we spelt it out for him)

“Excuse me….r u underestemating my intelegence here??? I have
entered the corect digits about 20 times!!!! u can check it
urself..fix the issue please..dont have tiome for this crap,still not
working.Sell valid vouchers to ur customers please,i spend allot of
money each month on u guys!!”