Colin posed a great question.
I've been thinking about it for several days and I've had some fleeting ideas. In particular, I wanted to respond to this quote:
experimenting, but by the time enough people and money are committed to the research, it feels more like a validation. I run into a lot of business pilots, but that seems different than a prototype.
Let's go with the definition that Diego proposed last week that a "prototype is nothing other than a single question, embodied." That kind of sings. I like it.
If that's the case, then I'd propose that:
A pilot is a collection of many prototypes, embodied all at once, that might get me fired.
When you design complex systems, the focus needs to be on the interfaces and integration not just the components. Drawing way back on my Systems Engineering background (go Hoos!), complex systems have normal accidents. Things are destined to fail. My brain is too small to think about things like complex experiment design or Taguchi methods or something. Most people who work in companies are the exact same way.
Maybe what makes pilots so hard are they are such an organizational minefield? Once you get to the stage of a "pilot" in most large-scale organizations there are a few things going on: lots of attention, lots of pressures, lots of questions and lots of money. Not everywhere, but that's the conventional wisdom. Now, I like piloting more than "flipping the switch" all at once. Especially when you really don't know what you're doing yet (and are honest enough to admit it). Small to learn, big to win.
Maybe there's a difference between what it means to be developing prototypes, running pilots, testing and learning in-market and what it means to be "living in Beta." When you've reached organizational learning Nirvana, you can ask single questions again. I took a minute to sketch something. Does this work?
