The Visionary or the Statistician: Who is Better at Product Development?

By Evan Baehr (blog:

Consider two very different approaches to product development:

  • Apple's Ron Johnson describes that Apple is about building a beautiful product or experience and then persuading users that they actually want it.
  • At Zynga, their math-driven product development cycle essentially says: if we can make one penny more by making the tractor have 6 pixel wheels instead of 5 pixel wheels, let's do it.
The former is about having a vision of adding value and meaning to the user... of contributing to their human flourishing... of capturing a valuable insight into what makes them happy and building something that fulfills a desire they may not have even known they had.  These product developers have vision, insight, and a deep understanding of people.  Think Ideo: hundreds of man hours spent on observation, questioning, sketching, prototyping... and eventually out comes a beautiful shopping cart.
The latter is about a huge array of A/B testing that constantly iterates based on the user feedback.  These product developers begin with a basic idea and turn it over to the user, allowing thousands or millions of them to lead the product in any direction the users desire.  (From the offline world), think Harrah's casino:  eye-grabbing blinking, beeping, waitresses, booze, dealers... all "A/B tested" to extract every last penny from casino goers.
On the surface, these two approaches seem incompatible.  Yet in a recent exchange Eric Ries helped make sense of how insightful vision can be combined with iterative, agile implementation.  I identified this dichotomy: if you are trying to build something users don't know they need, then it doesn't make sense to ask them what they want (cf. Henry Ford's quip: "Had I asked them what they wanted they would have said 'a faster horse.'")  Eric said it clearly: "Listen to your customers, but don't do what they say. Their feedback tells you about them, not about you."
In this integrated approach, the developer never becomes a mercenary--under the control of the user and meeting their every desire. They remain a visionary... but not an arrogant, ignorant one.  They begin with a dream... but really get to know the people for whom they want to dream to become a reality.  Observation and data collection is critical not in order to dictate design but rather to clarify and focus vision.  (Nate Gross reminded me of Doug Bowman's resignation letter from Google in which he wrote, "data eventually becomes a crutch for every decision, paralyzing the company and preventing it from making any daring design decisions.")
On a visit with Facebook COO Sheryl Sandberg, I laid out these two approaches and asked which Facebook pursues.  She said: "Both."  Zuck began with a vision of connecting the world; indeed this vision involved people sharing in ways that they current do not.  His vision is quite Apple-like in that he has an insight into what will make people flourish; even though users don't know they want this, over time they will realize it.  And Zuck's job is to bring them along on this journey.  Although the vision is clear, the implementation is not.  Facebook has to understand and learn from its users to find out exactly how to create deeply social and meaningful experiences.  So in effect Facebook began with a long-term vision to which they will lead users, yet the path to that vision is unclear.  Zuck has a vision... but gets to converse with 650 million users about how to get there. Perhaps Zuck has pulled off "doing both."
Yet it would be false to "do both" in the sense of being partly one and partly the other. Products created with a hint of vision and a hint of feedback are likely neither visionary nor useful.  Successfully combining the two approaches requires a keen understanding of the role of vision and the role of feedback.  Both Steve Jobs and Zuck do this incredibly well and are what I would call "ethnographer visionaries" - leaders whose profound product visions come not out of prophecy or momentary brilliance but rather out of a deep understanding of people.  In the realm of product management, I favor visionaries over zealots and statisticians over mercenaries.