VALIDATED LEARNING AT IMVU

 

Let me illustrate this with an example from my career. Many audiences have heard me recount the story of IMVU’s founding and the many mistakes we made in developing our first product. I’ll now elaborate on one of those mistakes to illustrate validated learning clearly.

Those of us involved in the founding of IMVU aspired to be serious strategic thinkers. Each of us had participated in previous ventures that had failed, and we were loath to repeat that experience. Our main concerns in the early days dealt with the following questions: What should we build and for whom? What market could we enter and dominate? How could we build durable value that would not be subject to erosion by competition?1

Brilliant Strategy

 

We decided to enter the instant messaging (IM) market. In 2004, that market had hundreds of millions of consumers actively participating worldwide. However, the majority of the customers who were using IM products were not paying for the privilege. Instead, large media and portal companies such as AOL, Microsoft, and Yahoo! operated their IM networks as a loss leader for other services while making modest amounts of money through advertising.

广告:个人专属 VPN,独立 IP,无限流量,多机房切换,还可以屏蔽广告和恶意软件,每月最低仅 5 美元

IM is an example of a market that involves strong network effects. Like most communication networks, IM is thought to follow Metcalfe’s law: the value of a network as a whole is proportional to the square of the number of participants. In other words, the more people in the network, the more valuable the network. This makes intuitive sense: the value to each participant is driven primarily by how many other people he or she can communicate with. Imagine a world in which you own the only telephone; it would have no value. Only when other people also have a telephone does it become valuable.

In 2004, the IM market was locked up by a handful of incumbents. The top three networks controlled more than 80 percent of the overall usage and were in the process of consolidating their gains in market share at the expense of a number of smaller players.2 The common wisdom was that it was more or less impossible to bring a new IM network to market without spending an extraordinary amount of money on marketing.

The reason for that wisdom is simple. Because of the power of network effects, IM products have high switching costs. To switch from one network to another, customers would have to convince their friends and colleagues to switch with them. This extra work for customers creates a barrier to entry in the IM market: with all consumers locked in to an incumbent’s product, there are no customers left with whom to establish a beachhead.

At IMVU we settled on a strategy of building a product that would combine the large mass appeal of traditional IM with the high revenue per customer of three-dimensional (3D) video games and virtual worlds. Because of the near impossibility of bringing a new IM network to market, we decided to build an IM add-on product that would interoperate with the existing networks. Thus, customers would be able to adopt the IMVU virtual goods and avatar communication technology without having to switch IM providers, learn a new user interface, and—most important—bring their friends with them.

In fact, we thought this last point was essential. For the add-on product to be useful, customers would have to use it with their existing friends. Every communication would come embedded with an invitation to join IMVU. Our product would be inherently viral, spreading throughout the existing IM networks like an epidemic. To achieve that viral growth, it was important that our add-on product support as many of the existing IM networks as possible and work on all kinds of computers.

Six Months to Launch

 

With this strategy in place, my cofounders and I began a period of intense work. As the chief technology officer, it was my responsibility, among other things, to write the software that would support IM interoperability across networks. My cofounders and I worked for months, putting in crazy hours struggling to get our first product released. We gave ourselves a hard deadline of six months—180 days—to launch the product and attract our first paying customers. It was a grueling schedule, but we were determined to launch on time.

The add-on product was so large and complex and had so many moving parts that we had to cut a lot of corners to get it done on time. I won’t mince words: the first version was terrible. We spent endless hours arguing about which bugs to fix and which we could live with, which features to cut and which to try to cram in. It was a wonderful and terrifying time: we were full of hope about the possibilities for success and full of fear about the consequences of shipping a bad product.

Personally, I was worried that the low quality of the product would tarnish my reputation as an engineer. People would think I didn’t know how to build a quality product. All of us feared tarnishing the IMVU brand; after all, we were charging people money for a product that didn’t work very well. We all envisioned the damning newspaper headlines: “Inept Entrepreneurs Build Dreadful Product.”

As launch day approached, our fears escalated. In our situation, many entrepreneurial teams give in to fear and postpone the launch date. Although I understand this impulse, I am glad we persevered, since delay prevents many startups from getting the feedback they need. Our previous failures made us more afraid of another, even worse, outcome than shipping a bad product: building something that nobody wants. And so, teeth clenched and apologies at the ready, we released our product to the public.

Launch

 

And then—nothing happened! It turned out that our fears were unfounded, because nobody even tried our product. At first I was relieved because at least nobody was finding out how bad the product was, but soon that gave way to serious frustration. After all the hours we had spent arguing about which features to include and which bugs to fix, our value proposition was so far off that customers weren’t getting far enough into the experience to find out how bad our design choices were. Customers wouldn’t even download our product.

Over the ensuing weeks and months, we labored to make the product better. We brought in a steady flow of customers through our online registration and download process. We treated each day’s customers as a brand-new report card to let us know how we were doing. We eventually learned how to change the product’s positioning so that customers at least would download it. We were making improvements to the underlying product continuously, shipping bug fixes and new changes daily. However, despite our best efforts, we were able to persuade only a pathetically small number of people to buy the product.

In retrospect, one good decision we made was to set clear revenue targets for those early days. In the first month we intended to make $300 in total revenue, and we did—barely. Many friends and family members were asked (okay, begged). Each month our small revenue targets increased, first to $350 and then to $400. As they rose, our struggles increased. We soon ran out of friends and family; our frustration escalated. We were making the product better every day, yet our customers’ behavior remained unchanged: they still wouldn’t use it.

Our failure to move the numbers prodded us to accelerate our efforts to bring customers into our office for in-person interviews and usability tests. The quantitative targets created the motivation to engage in qualitative inquiry and guided us in the questions we asked; this is a pattern we’ll see throughout this book.

I wish I could say that I was the one to realize our mistake and suggest the solution, but in truth, I was the last to admit the problem. In short, our entire strategic analysis of the market was utterly wrong. We figured this out empirically, through experimentation, rather than through focus groups or market research. Customers could not tell us what they wanted; most, after all, had never heard of 3D avatars. Instead, they revealed the truth through their action or inaction as we struggled to make the product better.

Talking to Customers

 

Out of desperation, we decided to talk to some potential customers. We brought them into our office, and said, “Try this new product; it’s IMVU.” If the person was a teenager, a heavy user of IM, or a tech early adopter, he or she would engage with us. In constrast, if it was a mainstream person, the response was, “Right. So exactly what would you like me to do?” We’d get nowhere with the mainstream group; they thought IMVU was too weird.

Imagine a seventeen-year-old girl sitting down with us to look at this product. She chooses her avatar and says, “Oh, this is really fun.” She’s customizing the avatar, deciding how she’s going to look. Then we say, “All right, it’s time to download the instant messaging add-on,” and she responds, “What’s that?”

“Well, it’s this thing that interoperates with the instant messaging client.” She’s looking at us and thinking, “I’ve never heard of that, my friends have never heard of that, why do you want me to do that?” It required a lot of explanation; an instant messaging add-on was not a product category that existed in her mind.

But since she was in the room with us, we were able to talk her into doing it. She downloads the product, and then we say, “Okay, invite one of your friends to chat.” And she says, “No way!” We say, “Why not?” And she says, “Well, I don’t know if this thing is cool yet. You want me to risk inviting one of my friends? What are they going to think of me? If it sucks, they’re going to think I suck, right?” And we say, “No, no, it’s going to be so much fun once you get the person in there; it’s a social product.” She looks at us, her face filled with doubt; you can see that this is a deal breaker. Of course, the first time I had that experience, I said, “It’s all right, it’s just this one person, send her away and get me a new one.” Then the second customer comes in and says the same thing. Then the third customer comes in, and it’s the same thing. You start to see patterns, and no matter how stubborn you are, there’s obviously something wrong.

Customers kept saying, “I want to use it by myself. I want to try it out first to see if it’s really cool before I invite a friend.” Our team was from the video game industry, so we understood what that meant: single-player mode. So we built a single-player version. We’d bring new customers into our office. They’d customize the avatar and download the product like before. Then they would go into single-player mode, and we’d say, “Play with your avatar and dress it up; check out the cool moves it can make.” Followed by, “Okay, you did that by yourself; now it’s time to invite one of your friends.” You can see what’s coming. They’d say, “No way! This isn’t cool.” And we’d say, “Well, we told you it wasn’t going to be cool! What is the point of a single-player experience for a social product?” See, we thought we should get a gold star just for listening to our customers. Except our customers still didn’t like the product. They would look at us and say, “Listen, old man, you don’t understand. What is the deal with this crazy business of inviting friends before I know if it’s cool?” It was a total deal breaker.

Out of further desperation, we introduced a feature called ChatNow that allows you to push a button and be randomly matched with somebody else anywhere in the world. The only thing you have in common is that you both pushed the button at the same time. All of a sudden, in our customer service tests, people were saying, “Oh, this is fun!”

So we’d bring them in, they’d use ChatNow, and maybe they would meet somebody they thought was cool. They’d say, “Hey, that guy was neat; I want to add him to my buddy list. Where’s my buddy list?” And we’d say, “Oh, no, you don’t want a new buddy list; you want to use your regular AOL buddy list.” Remember, this was how we planned to harness the interoperability that would lead to network effects and viral growth. Picture the customer looking at us, asking, “What do you want me to do exactly?” And we’d say, “Well, just give the stranger your AIM screen name so you can put him on your buddy list.” You could see their eyes go wide, and they’d say, “Are you kidding me? A stranger on my AIM buddy list?” To which we’d respond, “Yes; otherwise you’d have to download a whole new IM client with a new buddy list.” And they’d say, “Do you have any idea how many IM clients I already run?”

“No. One or two, maybe?” That’s how many clients each of us in the office used. To which the teenager would say, “Duh! I run eight.” We had no idea how many instant messaging clients there were in the world.

We had the incorrect preconception that it’s a challenge to learn new software and it’s tricky to move your friends over to a new buddy list. Our customers revealed that this was nonsense. We wanted to draw diagrams on the whiteboard that showed why our strategy was brilliant, but our customers didn’t understand concepts like network effects and switching costs. If we tried to explain why they should behave the way we predicted, they’d just shake their heads at us, bewildered.

We had a mental model for how people used software that was years out of date, and so eventually, painfully, after dozens of meetings like that, it started to dawn on us that the IM add-on concept was fundamentally flawed.3

Our customers did not want an IM add-on; they wanted a stand-alone IM network. They did not consider having to learn how to use a new IM program a barrier; on the contrary, our early adopters used many different IM programs simultaneously. Our customers were not intimidated by the idea of having to take their friends with them to a new IM network; it turned out that they enjoyed that challenge. Even more surprising, our assumption that customers would want to use avatar-based IM primarily with their existing friends was also wrong. They wanted to make new friends, an activity that 3D avatars are particularly well suited to facilitating. Bit by bit, customers tore apart our seemingly brilliant initial strategy.

Throwing My Work Away

 

Perhaps you can sympathize with our situation and forgive my obstinacy. After all, it was my work over the prior months that needed to be thrown away. I had slaved over the software that was required to make our IM program interoperate with other networks, which was at the heart of our original strategy. When it came time to pivot and abandon that original strategy, almost all of my work—thousands of lines of code—was thrown out. I felt betrayed. I was a devotee of the latest in software development methods (known collectively as agile development), which promised to help drive waste out of product development. However, despite that, I had committed the biggest waste of all: building a product that our customers refused to use. That was really depressing.

I wondered: in light of the fact that my work turned out to be a waste of time and energy, would the company have been just as well off if I had spent the last six months on a beach sipping umbrella drinks? Had I really been needed? Would it have been better if I had not done any work at all?

There is, as I mentioned at the beginning of this chapter, always one last refuge for people aching to justify their own failure. I consoled myself that if we hadn’t built this first product—mistakes and all—we never would have learned these important insights about customers. We never would have learned that our strategy was flawed. There is truth in this excuse: what we learned during those critical early months set IMVU on a path that would lead to our eventual breakout success.

For a time, this “learning” consolation made me feel better, but my relief was short-lived. Here’s the question that bothered me most of all: if the goal of those months was to learn these important insights about customers, why did it take so long? How much of our effort contributed to the essential lessons we needed to learn? Could we have learned those lessons earlier if I hadn’t been so focused on making the product “better” by adding features and fixing bugs?