Crowdsourcing “in reverse”: asking crowds to ask questions

It’s important to understand that crowdsourcing is first and foremost a question, a question that you ask a large and, ideally, diversified crowd of people. And for as long as it’s well-thought-out, properly defined, and clearly articulated, it doesn’t really matter what this question is about. It can be a question about a solution to a problem, something crowdsourcing is mostly known for; it can also be a question about the problem itself.

I call asking questions about a problem crowdsourcing “in reverse.” A few years ago, researchers at Harvard Medical School proved the effectiveness of this approach. They asked the crowd the following question: what do we not know to cure Type 1 diabetes? In other words, are there “neglected” problems that for whatever reasons were off the radars of the existing Type 1 diabetes research groups? What questions need to be asked to accelerate the rate of Type 1 diabetes research?

Interestingly, among 12 winning contributions, there was one submitted by a diabetes patient. Although lacking appropriate scientific background, this person has provided a unique perspective on the type of challenges faced by diabetes patients, a perspective that can’t be offered by a healthy individual.

Such approach—using crowdsourcing to combine scientific knowledge of doctors with experiential knowledge of patients—has been expanded and further developed by researchers at the Open Innovation in Science Center in Vienna, Austria. They addressed the issue of mental health and illness, a subject that is highly relevant from the public health, economic and policy points of view, yet relatively under-researched when compared to other medical conditions (such as cancer, for example).

During the submission period, which lasted for 11 weeks in spring-summer 2015, patients, their caregivers, doctors, and other medical professionals were asked to highlight unresolved problems and open research questions in the field of mental health. Characteristically, 40% of the received contributions were submitted by people who described themselves as patients.

The crowdsourcing campaign has identified an area of mental health research that hasn’t received sufficient attention so far: mental health of children and adolescents with mentally ill parents. To fill the gap, two research projects addressing this issue have since been launched with a total funding of six million euros over the period of four years.

The Center’s next target is the field of orthopedic traumatology. By launching a crowdsourcing campaign called “Tell Us!”, the researchers want to generate novel and original research questions, both from experts and patients, that have previously not been properly addressed in the area of traumatology research. As with the previous project, all “out-of-the-box” questions, ideas and hypotheses will be fed back into the scientific workflow.

The project will begin in early May 2018 and last for two months. If you believe (as I do) that this particular way of using crowdsourcing makes a lot of sense, please, help spread the word about the project. The project website is

I’d like to thank Benjamin Missbach (, Project Manager at the Open Innovation in Science Center, for introducing me to the “Tell Us!” project. I’m also grateful to him for his comments on this piece.

The image credit:

p.s. To subscribe to my monthly newsletter on crowdsourcing, go to

Posted in Crowdsourcing, Health Care | Tagged , , , , , , , | 6 Comments

Crowdsourcing 2.0

I like to argue, only half-jokingly, that crowdsourcing is very simple. It consists of only two components: a question and a crowd—a question that you present to a crowd and a crowd that you assemble to answer this question.

And yet, the acceptance of crowdsourcing as a practical problem-solving tool has been slow. The explanation might be quite simple: as every tool, crowdsourcing requires certain knowledge and skills to be properly used–and it is this knowledge and skills that are lacked by many organizations.

To begin with the question “part,” there is almost a paralyzing uncertainty over the issue of which technical or business problem can be successfully solved by crowdsourcing. Besides, organizations that are new to crowdsourcing often struggle with the crowd “part,” too, as they feel profoundly confused with a vast variety of available crowdsourcing platforms. Mistakes are very common when novice users try to match their problems with a specific, most suitable platform.

More time and effort will be needed to make the basics of crowdsourcing—Crowdsourcing 1.0, so to speak—a common knowledge among practitioners in the field. At the same time, it’s never too early to start imagining what the future of crowdsourcing—I’d call it Crowdsourcing 2.0—may look like.

I’m particularly interested in a role artificial intelligence (AI) will play in molding the parameters of crowdsourcing of tomorrow. On the “question front,” a large amount of data is already available describing both successful and failed crowdsourcing campaigns. Analyzing the data will help identify types of problems—and/or essential components and features of them–that make any problem more (or less) amenable to solving by applying the wisdom of crowds. Moreover, it’ll be certainly possible to design machine algorithms allowing users to translate their technical or business challenges into specific problem statements with the highest expected levels of success. The same algorithms will tell the users which kind of information they should be ready to provide to the crowd to maximize the odds for the problem to be solved.

AI has a potential to shape the process of matching a specific problem to a “perfect” crowdsourcing platform, too—although this may require some additional effort, given that existing commercially available platforms are often short-lived and have a poor track record of success. By analyzing a set of problems a platform has dealt with in the past, the reported solution rate, and the size and the composition of the crowd behind the platform (where this info is available), it’ll be eventually possible to automatically generate a list of the most promising platforms to tackle any particular problem. AI tools that will allow a rapid creation of sufficiently large and diverse crowds (“crowds-on-demand”) are expected to be developed, too.

As a quote credited to both Abraham Lincoln and Peter Drucker puts it: the best way to predict the future is to create it. The creation of the future of crowdsourcing (Crowdsourcing 2.0) begins today.

p.s. To subscribe to my monthly newsletter on crowdsourcing, go to

The image credit:

Posted in Crowdsourcing | Tagged , , , , , , | Leave a comment

Are you innovating? We won’t be paying you today!

A solid body of evidence, from both controlled laboratory experiments and field studies, shows that compensation based on the pay-for-performance (P-f-P) principle—when individuals receive a fixed percentage of the profits resulted from their activities–is effective in inducing higher levels of effort and productivity. However, one must keep in mind that this evidence comes from studying simple routine tasks, in which effort is the main prerequisite for productivity.

But what about activities such as innovation, which requires creativity and exploration of novel approaches? The concern here is that P-f-P, when applied to innovation, could undermine performance because it’ll encourage repetition of what has worked in the past, rather than experimenting with new ways of doing things.

How should executive and managerial compensation be structured if the goal is to foster innovation?

An interesting insight was provided by Florian Ederer and Gustavo Manso in their 2013 paper “Is Pay-for-Performance Detrimental to Innovation?” Ederer & Manso conducted a series of laboratory experiments, in which 379 subjects faced the choice between exploitation and exploration. More specifically, subjects were asked to control the operations of a fictional lemonade stand and had to find the optimal location of the stand as well and the optimal product features. Subjects had a choice to either fine-tune the recommendations given to them by the “previous manager” (exploitation) or select an entirely different strategy (exploration).

There were three different treatment groups, with the only difference between them being the structure of compensation. Subjects in the first treatment group received a fixed wage during the whole experiment. Subjects in the second group were given a standard P-f-P contract (a fixed percentage of the profits produced during the experiment). For the subjects in the third group, the compensation was a fixed percentage of the profits produced during the second half of the experiment. This compensation structure allowed the subjects to fail at no cost in the first half of the experiment while exploring new strategies, but reap the full benefits of their exploration after learning better ways of performing the task.

The authors’ main hypothesis was that subjects in the third group would explore more than subjects in two other groups while trying to find a superior strategy. Experimental results did confirm this assumption. Moreover, the results showed that the subjects in the third group not only explored more; they showed superior performance in finding an optimal performance strategy.

The authors found that risk aversion played a key role in accounting for the poor performance of the subjects in the second (P-f-P contract) group: under the P-f-P contract, more risk-averse subjects were less likely to find the optimal performance strategy; as the result, they obtained lower average profits than less risk-averse subjects.

The above study provides a solid experimental support to a theoretical analysis conducted by Manso in 2011 that showed that the optimal incentives motivating employees to innovate must include a combination of tolerance for failures in the short term and reward for success in the long run. Tolerance for early failures allows the employees to take risks at the initial stages of the innovation process without incurring the negative consequences of failed projects. In a series of previous posts, I discussed how tolerance for innovation failures can be institutionalized by the application of the wrongful discharge laws, debtor-friendly bankruptcy codes, and choosing risk-tolerant VC investors.

The reward for long-term success encourages the employees to explore risky ideas that may allow them to achieve innovation breakthroughs in more distant future. The best way of achieving such an encouragement seems to be via compensation strategies heavily relying on long-term stock option grants. I’ll return to this specific topic in my future posts.

p.s. To subscribe to my monthly newsletter on crowdsourcing, go to

The image credit:

Posted in Innovation | Tagged , , , , , , | 1 Comment

One more time about “innovation terminology”

In a recent HBR article, Scott Kirsner suggests ditching the term “corporate entrepreneur.”

Kirsner names a number of reasons why corporate innovation, especially in large firms, is different from true entrepreneurship. One is bureaucratic shackles that restrict the development of new big ideas in a corporation; another is an obsession with short-term performance goals that prevents large companies from pursuing risky long-term projects. Kirsner goes as far as to argue that using the very term “entrepreneur” would set up corporate innovators for failure in a large organization.

Although I agree with Kirsner that the term “corporate entrepreneur” (along with, I would add, the equally ambiguous term “intrapreneur”) is more confusing than clarifying, just getting rid of it will not help cure the maladies of corporate innovation.

The real problem with corporate innovation is that some corporate leaders are still unsure what innovation really is. Many organizations have no working definition of what innovation means for them, and corporate innovation is routinely equated with occasional “idea generation” campaigns.

There is a widespread lack of understanding of the difference between incremental, adjacent and transformational innovation (a.k.a. the 3-Horizon Model of Innovation); very few are familiar with the concept of Integrative Innovation Management. Many corporate innovators sincerely believe that every innovation must be “disruptive” and that all other types of it are for losers.

There is simply no reason to invoke “corporate entrepreneurship” when operating within the borders of incremental and adjacent innovation; the needs of these two types of innovation can be adequately addressed by the conventional product development processes augmented, if needed, by open innovation approaches, such as customer co-creation.

It’s only when organizations deal with transformational innovation do they really need an infusion of “entrepreneurial” spirit. However, the best way of doing that would be by engaging startups, the entities that are being created for the sole purpose of transformational change of the existing technology and business landscape. By engaging startups, organizations would “hire” the very people capable of challenging the corporate status quo–without going through a murky process of creating “corporate entrepreneurship.”

The bottom line is that corporate innovation requires strategy, governance, processes, and control. Replacing the basics with catchy buzzwords won’t cut it.

p.s. To subscribe to my monthly newsletter on crowdsourcing, go to

The image credit:

Posted in Innovation | Tagged , , , , , , , , , , , , , , | Leave a comment

Freedom to fail, freedom to innovate

(This post originally appeared on

Everyone seems to agree that innovation is a risky business: it involves a lot of experimentation, which often ends up in failure. High tolerance for failure, therefore, can be considered as a major prerequisite for any successful innovation program.

While talks about accepting or even “celebrating” failures are becoming commonplace, little attention is being paid to how positive attitude towards failure should be promoted. Which specific policies could be put in place to make safe–and, better yet, attractive–for firms and their employees to engage in innovation activities?

The “labor law” of innovation

Academic research provides useful guidance with respect to what these policies should be. Theoretical analysis by Gustavo Manso suggests that conditions incentivizing employees to innovate must include tolerance for early failures, which would allow them to take risks at the initial stages of the innovation process without facing negative consequences of failed projects.

Available empirical evidence supports Manso’s conclusion. For example, an analysis of the impact of labor laws on innovation in five countries showed that stronger labor laws positively correlated with a country’s innovation output. In another study, the impact on innovation of the U.S. wrongful discharge laws (WDL) was investigated. These laws provide employees with greater protection than employment-at-will, under which they can be terminated with or without just cause. The WDL, particularly those that protect employees from termination in bad faith, were found to foster innovation by increasing the employees’ motivation and effort.

These results strongly suggest that innovation is promoted by laws that limit firms’ ability to discharge employees at will. Experts call this phenomenon an “insurance effect”: feeling increased protection from negative consequences of failure, employees are more committed to engaging in risky innovative projects.

Innovation and bankruptcy laws

It turns out that corporate innovation is encouraged not only by protecting individual employees but the firms, too. A study of bankruptcy laws in 12 countries showed that more debtor-friendly bankruptcy codes (i.e., codes favoring firms filing for bankruptcy) had a positive effect on corporate innovation. The debtor-friendly laws encourage firm-level innovation by promoting the continuation of innovative activities even following the firm’s bankruptcy.

Innovation and investment

Another piece of evidence that tolerance for failure positively affects corporate innovation comes from a study investigating how venture capital (VC) investors deal with failures of startup firms they fund. Startups have high innovation potential, but they also carry a substantial risk of failure; innovation failure tolerance is therefore very important to them. That’s why VC investors’ own tolerance for failure is crucial. If they can’t tolerate early failures, they would prematurely liquidate a startup upon initial unsatisfactory progress and thus prevent it from realizing the startup’s true innovation potential.

The study shows that startups backed by more tolerant VC investors are significantly more innovative. The effect of VC failure tolerance on startup innovation is much stronger when the failure risk is higher and thus failure tolerance is more needed and valued. For example, being financed by a failure-tolerant VC is much more important for ventures born in recessions or ventures at early development stages.

Practical steps

The results of the above studies suggest that firms can increase the efficiency of their corporate innovation by simply modifying its termination policies. For example, they can place employees involved in strategic innovation projects on fixed-term employment contracts (as opposed to employment-at-will). Alternatively, tenure-like positions may be created for the same employees.

Admittedly, capitalizing on the effects of international bankruptcy laws and VC investors’ tolerance for failure isn’t straightforward. However, firms should consider local bankruptcy codes when choosing the location of their innovation centers. And startup companies ought to be aware of the failure tolerance level of VC investors they choose.

A larger point, however, is that we must finally move from words to deeds when dealing with innovation failures. Providing people with immunity from failures is better than “celebrating” them.

The image was provided by Tatiana Ivanov

Posted in Innovation | Tagged , , , , , | 2 Comments

Innovation and VC investors

Academic research provides abundant empirical evidence suggesting that corporate and socio-economic policies tolerating failure, both at individual and firm levels, foster innovation. For example, labor laws limiting firms’ ability to discharge employees at will were shown to stimulate corporate innovation. Corporate innovation is encouraged not only by protecting individual employees but the firms themselves, as evidenced by the finding that debtor-friendly (i.e., firm-friendly) bankruptcy laws have a positive effect on innovation output.

Yet another piece of evidence that the tolerance for failure positively affects corporate innovation comes from the work by Xuan Tian and Tracy Yue Wang published in 2011.

Tian and Wang studied the subject of innovation failures using an original empirical approach: by analyzing venture capital (VC) investors’ attitude towards failure in VC-backed startup firms. Startup firms are known to have high innovation potential, but also a substantial failure risk; thus, innovation failure tolerance is very important to them. On the other hand, VC investors’ tolerance for failure is crucial for the innovation productivity of VC-backed startups. If VC investors can’t tolerate early failures, they can prematurely liquidate a startup upon initial unsatisfactory progress and prevent it from realizing the startup’s true innovation potential.

Tian and Wang developed a practical measure of VC investor’s failure tolerance: they examined investors’ willingness to continue funding underperforming ventures (i.e., ventures not meeting pre-determined milestones). The rationale behind this approach is that when a project does not show progress towards its goals, a VC investor has a choice: to give the entrepreneur a second chance by continued funding or to write off the project immediately. What a choice the VC made would to some extent reflect their attitude towards failure. Other things equal, the longer the VC firm waits before terminating funding of underperforming projects, the more tolerant the VC is for early failures in investments.

Tian and Wang showed that IPO firms backed by more tolerant VC investors are significantly more innovative in terms of the quantity and quality of patents issued to them. (More specifically, if a VC firm on average invests for two years before terminating a project, but is willing to invest for additional two years in an IPO firm, then this IPO firm tends to have 40% more patents per year later.)

Tian and Wang also found that the effect of VC failure tolerance on startup innovation was much stronger when the failure risk is higher and thus failure tolerance is more needed and valued. For example, being financed by a failure-tolerant VC is much more important for ventures born in recessions, ventures at early development stages, and ventures in industries in which innovation is especially difficult (e.g., drug development). VCs’ tolerance for failure allows these startups to overcome early difficulties and realize their true innovation potential.

In my previous post, I described a Stanford study that shows that firms going for IPO may jeopardize their innovation output. In fact, going public can result in a 40% decline in the quality of patents issued to the IPO firm in the five years immediately following listing. Of course, this fact by itself shouldn’t prevent startups from considering IPO. However, they ought to be mindful of the consequences this development may have on their post-IPO ability to innovate—and they must adjust their corporate innovation policies accordingly. Choosing a “correct” (i.e., failure-tolerant) VC investor would be a smart move.

p.s. To subscribe to my monthly newsletter on crowdsourcing, go to

The image credit:

Posted in Innovation | Tagged , , , , , | 2 Comments

Innovation and bankruptcy

Recently, I described academic studies suggesting that corporate innovation is fostered by labor laws that limit firms’ ability to discharge employees at will. These studies provide support to the idea that the best way to encourage risk-taking and experimentation is not to “celebrate failures,” but to remove the proverbial Sword of Damocles of punishment for them, something that any organization can do by modifying its termination policies.

It turns out that corporate innovation is encouraged not only by protecting individual employees but the firms themselves. In a paper published in 2009, “Bankruptcy Codes and Innovation,” Viral Acharya and Krishnamurthy Subramanian show that firm-friendly bankruptcy laws have a positive effect on corporate innovation.

Acharya & Subramanian have analyzed changes in bankruptcy codes that took part in 12 countries in 1978-2002. Over this period, seven countries (Canada, Finland, Indonesia, Ireland, India, Israel, and Sweden) have made their bankruptcy codes more debtor-friendly, i.e., favoring firms filing for bankruptcy. Five countries (Denmark, UK, Lithuania, Romania, and the Russian Federation) have made their country codes creditor-friendly, i.e., giving more rights to firms’ creditors.

Compared to a “control” group of counties—those that did not undergo a creditor rights change–the “treatment” group of five countries that underwent a firm’s rights decrease demonstrated a 9.7% decrease in the number of patents issued to these countries and a 13.3% decrease in their quality. In contrast, in seven countries where firm’s rights were increased, the number of patents rose by 10.7% and their quality by 15.4%.

There are two major factors contributing to creditor-friendly bankruptcy codes’ negative effect on innovation. First, they lead to the excessive liquidation of firms’ assets, which results in the reduction of funds available for innovation activities. Second, in countries with stronger creditor rights, innovative firms take smaller quantities of debt and keep more cash reserves, which again limits their ability to invest in innovative projects.

The major conclusion from the data presented by Acharya & Subramanian is that debtor-friendly bankruptcy codes encourage firm-level innovation by promoting the continuation of innovative activities even following the firm’s bankruptcy. Saying the same thing differently, very much like their employees, firms, too, need protection from failure.

p.s. To subscribe to my monthly newsletter on crowdsourcing, go to

The image credit:

Posted in Innovation | Tagged , , , | 6 Comments