What’s the deal with Win/Loss Analysis?

Last week’s poll about Win/Loss Analysis has pointed out some grim facts about us as product managers. To summarize:

  • According to the Pragmatic Marketing survey, only about 20% of product managers perform Win/Loss Analysis.
  • Comments and informal conversations suggest that 20% is too high. What does it mean to answer in the affirmative, i.e., “I do win/loss analysis”? How much analysis? How many customer interviews? How many loss interviews? Using what method or technique?

To explore further, we conducted a poll to ask why. The results were posted last week. The largest single response was “other” (29%), and many of those responses said essentially, “Sales won’t let me”.

If most of the 29% of people said they are blocked by sales, and 14% of responses said “Sales does it”, we have a major problem: For about 43% of you, Sales owns Win/Loss analysis or prevents Win/Loss from being done.

But let’s dig deeper. A further 10% said “it’s not considered important in my company”, and 24% said “Got way too many other things to do.”
Add those numbers to our earlier 43%, and that means that for 77% of our respondents, Win/Loss is not really your job.

win-loss avoidance reasons

Reasons you avoid Win/Loss Analysis

(note: I’ve changed the “reasons” from the original question to point out what’s really going on. I also re-ordered the list. I changed “other” to “Sales prevents PM from doing Win/Loss” because nearly all of those responses could be summarized that way. I’m illustrating reality, not aiming for statistical perfection.)

What’s going on here?
The table above lists the symptoms of the problem, but my diagnosis is this:

  • Product Managers are not doing much Product Management.
  • Companies like Product Managers, but do not fund or require Product Management.

This is a sorry state of affairs, folks.

You’re not doing Product Management

We are calling for a Product Management Manifesto to respond to this situation. But in the mean time, I’d like to point out a few common traps that might indicate a problem. With apologies to Jeff Foxworthy:

  • if your prospect calls are also sales calls, you’re not doing product management
  • if win/loss analysis means tabulating sales notes in the CRM, you’re not doing product management
  • if there is an SE in the room, and you’re doing the demo, you’re not doing product management
  • if your lawn furniture used to be your living room furniture, you might be a red neck (couldn’t resist)
  • if you think primary research means speaking directly with the sales team, you’re not doing product management
  • if you read analyst reports to understand market direction, you’re not doing product management
  • if you think competitive analysis means summarizing your competitors’ websites, you’re not doing product management
  • if “enhancement requests” are your primary source of product requirements, you’re not doing product management

And finally,

  • if your CEO trusts your sales people for what’s really happening in the market, you’re not doing product management.

In all seriousness, it’s time to change! I don’t know how you can effectively steer the ship if you’re not doing regular Win/Loss Analysis.

Next up: how NOT to do Win/Loss Analysis



10 responses to “What’s the deal with Win/Loss Analysis?

  1. Hi,

    I for one just was unable to find time to do the Win/Loss calls. When i FINALLY got to doing it. TRUST me it was boring / not inclined to do in the beginning.

    BUT at the end of the day when i need to know WHAT is missing in my product / roadmap it helped me a lot. I use salesforce to send mailers to a lot of contacts [saves me time]

    Product Lead

  2. Just curious, before you tell me I’m not doing (your perception of) product management, do you believe that the data on which you are drawing these conclusions is even statistically relevant?

  3. Jitin, thanks for your response. I’m interested to know what you found boring in the beginning about Win/Loss calls. Personally I find it invigorating to call both wins and losses. They give me “currency” to help set direction for the product line, and they give me authority with executive management and the board of directors.

    Re your comment about salesforce: I’m all for the idea of using salesforce to send mailers to initiate contact. As I’ll write about soon, many PMs use salesforce as the primary source of win/loss reasons. This approach is flawed.

  4. John, what kind of product management are you doing? What is your perception of product management?

    As for the data. The primary statistic that I referred to was that only 20% of PMs “do win/loss analysis. This stat comes from the Pragmatic Marketing survey (http://www.pragmaticmarketing.com/publications/survey/2008/2008-annual-product-management-and-marketing-survey). Pragmatic conducts the survey every year. Here’s the statement they provide about statistical validity:

    > Over 1,100 responded to the survey,
    > which was conducted from November 3
    > through November 26, 2008 using Vovici’s
    > EFM Feedback.

    The rest of my post was trying to understand why only 1 in 5 Product Managers perform Win/Loss Analysis, according to the Pragmatic data, which comes from the only survey of its kind that I am aware of. For this we conducted a poll on our blog.

    Beyond the data, some anecdotal experience. In 15 years of product management work, I can count on one hand the number of PMs outside of people I have personally managed, who performed good win/loss analysis. In my own case I admit to getting comfortable and letting it slip once I felt I understood the market.

    But there are few tools that are as powerful as good Win/Loss analysis.

    John, I got the sense from your post that your reaction is feeling indignant or insulted. If that’s the case, please make an argument in the opposition. Give us some of your own statistics, or some examples from your experience that contradict what I’m suggesting.


  5. Hi Alan,

    Towards the begining i did not find it at all important. only when i started to force myself i realized the importance, I use salesforce email template to shot mailers every week. Have a list of questions which i ask when i call the prospect / customer. Uopdate salesforce with my findings + my product roadmap which i maintain in excel.

    I use a basic scoring system + market segmentation to then use the data.

    Alan..im ALL for win loss now am working on bettering my product roadmap…

    Will send you the excel file which i have maybe you can add / advice 🙂


  6. Thanks Jitin. I agree with you – Win/Loss analysis can be an incredibly powerful tool. I’d love to look at your excel file.

  7. Pingback: Win/Loss Analysis: What to do if you’re not allowed to call customers « On Product Management

  8. Pingback: Contacting lost accounts « On Product Management

  9. Pingback: How to get a lost account to speak with you « On Product Management

  10. Pingback: Happy (belated) birthday to us (again)! « On Product Management