Experimental Notes is a quarterly newsletter bringing you fresh research insights from experiments in innovation, strategy and entrepreneurship. Edited by Sharique Hasan, Hyunjin Kim and Rembrand Koning in collaboration with the Innovation Growth Lab. 

Hi there, 

Welcome to Experimental Notes. This newsletter is a collaboration between the Innovation Growth Lab and the Workshop on Field Experiments in Strategy, Innovation, and Entrepreneurship (and, yes, we're planning a 2022 conference!). Our quarterly newsletter will feature recent papers from across the social sciences that use RCTs to shed light on how to improve innovation, broaden the benefits of entrepreneurship and build firm strategies and growth. 

This is our very first issue, so please forward it to your friends, post to social media, and encourage others to sign up here.

For this issue, we received 12 field experiments across our three topic areas. If you have a paper published recently (congrats!), a new working paper you're excited to share with the world, or recently read an excellent paper that you think others should know about, then please send them over to us! 

On entrepreneurship, we have five papers that study the
following questions: 

  • What types of training lead people to become entrepreneurs? Training in entrepreneurial skills and individual coaching may matter more than leadership skills.
  • When does a "scientific approach to entrepreneurship" help start-ups most? When these start-ups are at more advanced stages. 
  • Is persistence or adaptability better for entrepreneurial success? Persistence in the short term; adaptability in the long term. 
  • How should entrepreneurs form their teams? With friends who have the necessary skills.
  • How does autonomy in choosing ideas or teammates affect start-up performance? Autonomy in one dimension (especially ideas) is better than neither or both.

We also received two innovation papers, including studies that ask:

  • How do the decisions of other evaluators affect the evaluation of novel ideas? When raters see others’ negative ratings, they begin to look for weaknesses. When they see positive ratings, they become more confident in their judgement.  
  • What types of incentives lead to better innovations? Winner-takes-all incentive structures lead to higher quality innovative outcomes.  

Finally, we received five strategy-focused papers, all related to managing human capital:

  • How does AI-generated feedback affect employee performance? AI improves the quality of feedback, but employees (especially those who are new) trust it less. 
  • How does the congruence in leadership and gender-typed claims by businesses affect recruitment? Female candidates prefer congruence.
  • How do organisational values affect employee misconduct? Communicating values decreases employee misconduct, but not when they are told they are being monitored.
  • Do firms benefit from communicating their stance on socio-political issues? There is little discernible benefit when employees agree, but a demotivating effect when employees disagree.
  • Do the career concerns of recruiters affect firm-worker matching? It does, and it's not good for the firm.

Thanks for sharing your work. If you have a paper that you want to circulate, drop us a line (and your article). 

Until next time,
Sharique, Hyunjin, Rem and the IGL team



Impact measurement based on repeated randomized control trials: The case of a training program to encourage social entrepreneurship by Thomas Åstebro, Florian Hoos

Strategic Entrepreneurship Journal

Designing effective entrepreneurship training programs is still a challenge despite the investments in training made by governments and private institutions, and its importance for economic growth. We report a case of impact measurement of a social entrepreneurship program based on repeated randomized controlled trials (RCTs), discuss challenges of conducting repeated RCTs, and implications for policy evaluation. Impact measures from the first edition of the program showed no detectable treatment effects. The second edition was adjusted by reducing leadership training and increasing traditional entrepreneurial skills training, and had strong treatment effects on entrepreneurial activities, the creation of a new venture during the program, and subsequent start-up activity. Employing sequential field experiments can improve entrepreneurship training programs despite the challenges of executing RCTs in the field.

When do Entrepreneurs Benefit from Acting Like Scientists? A Field Experiment in the UK by Elena Novelli; Chiara Spina

INSEAD Working Paper

Prior research suggests that firms in entrepreneurial settings benefit from a scientific approach to decision making that combines cognitive and evidence-based components. But to what extent and under what conditions is the scientific approach to decision-making associated with superior performance? To address this question, we conducted a field experiment with 261 UK entrepreneurs at different stages of business development, training half of them on a scientific approach to decision-making. Our results show that firms make the most of scientific decision-making when they are at a more advanced stage of development, as they generate higher revenues and productivity. We elaborate on the mechanisms behind this result and the implications for future research.

For startups, adaptability and mentor network diversity can be pivotal: Evidence from a randomized experiment on a mooc platform by Charles Eesley, Lynn Wu

MIS Quarterly

Entrepreneurs leading digital ventures are often advised to be adaptable. However, research on how to pursue adaptable strategies and whether such strategies improve short- or long-term digital venture outcomes is sparse. By utilizing the ability to control content presentation and to measure outcomes through a course using a MOOC platform, we can introduce exogenous variation in strategies and mentorship characteristics, and link these attributes to venture outcomes over time. Contrary to expectations, we find that minimizing adaptability by adhering to a strong, persistent vision often results in better short-term outcomes as measured by quality of the pitch in digital startups. It also, however, results in worse long-term outcomes as measured by revenue, funding, and pivoting to a new venture. A more adaptable approach, when combined with a mentor who can facilitate this strategy by providing access to a structurally diverse social network, can offer the best combination of short- and long-run outcomes. The results suggest that guidance on mentor selection—especially selecting for the mentor’s social network attributes—is important over time for reaping the benefits of an adaptable strategy, particularly for digital ventures at their early-stage.

Forming Entrepreneurial Teams: Mixing Business and Friendship to Create Transactive Memory Systems for Enhanced Success by Moran Lazar, Ella Miron-Spektor, Gilad Chen, Brent Goldfarb, Mia Erez and Rajshree Agarwal

Academy of Management Journal

Successfully navigating through critical uncertainties during the incipient stages requires new ventures to develop learning systems, and building the right team may be key in this process. Drawing on prior work indicating that entrepreneurial teams form using either an interpersonal-attraction strategy (relationships with similar others in a close network) or a resource-seeking strategy (instrumental focus on complementary skills), we theorize that a dual formation strategy, although challenging to execute, is critical for early performance. Using dual formation strategies from the onset fosters the development of stronger transactive memory systems, because close relationships facilitate smooth coordination among founders specializing in complementary tasks. Transactive memory systems thus mediate the relationship between formation strategies and early entrepreneurial success. Findings from two field observational studies and a field intervention study support our theory: teams formed based on a dual strategy raised greater seed funding on Kickstarter – a leading crowdfunding platform (Study 1), were more successful in a prestigious entrepreneurial competition (Study 2), and gained more profits from selling their initial products (Study 3). Our research advances knowledge on entrepreneurial team formation and offers practical recommendations to facilitate this process at such nascent, but critical stages.

Organizing Entrepreneurial Teams: A Natural Field Experiment on Autonomy over Choosing Teams and Ideas by Viktoria Boss; Linus Dahlander; Christoph Ihl; Rajshri Jayaraman

Forthcoming Organization Science

Scholars have suggested that autonomy can lead to better entrepreneurial team performance. Yet, there are different types of autonomy and they come at a cost. We shed light on whether two fundamental organizational design choices—granting teams autonomy to (1) choose project ideas to work on and (2) choose team members to work with—affect performance. We run a natural field experiment involving 939 students in a lean startup entrepreneurship course over 11 weeks. The aim is to disentangle the separate and joint effects of granting autonomy over choosing teams and choosing ideas compared to a baseline treatment with pre-assigned ideas and team members. We find that teams with autonomy over choosing either ideas or team members outperform teams in the baseline treatment as measured by pitch deck performance. The effect of choosing ideas is significantly stronger than the effect of choosing teams. However, the performance gains vanish for teams that are granted full autonomy over choosing both ideas and teams. This suggests the two forms of autonomy are substitutes. Causal mediation analysis reveals that the main effects of choosing ideas or teams can be partly explained by a better match of ideas with team members’ interests and prior network contacts among team members, respectively. While homophily and lack of team diversity cannot explain the performance drop among teams with full autonomy, our results suggest that self-selected teams fall prey to overconfidence and complacency too early to fully exploit the potential of their chosen idea. We discuss the implications of these findings for research on organizational design, autonomy, and innovation.


Conservatism Gets Funded? A Field Experiment on the Role of Negative Information in Novel Project Evaluation by Jacqueline N. Lane, Misha Teplitskiy, Gary Gray, Hardeep Ranu, Michael Menietti, Eva C. Guinan, and Karim R. Lakhani

Management Science

The evaluation and selection of novel projects lies at the heart of scientific and technological innovation, and yet there are persistent concerns about bias, such as conservatism. This paper investigates the role that the format of evaluation, specifically information sharing among expert evaluators, plays in generating conservative decisions. We executed two field experiments in two separate grant funding opportunities at a leading research university, mobilizing 369 evaluators from seven universities to evaluate 97 projects resulting in 761 proposal-evaluation pairs and over $300,000 in awards. We exogenously varied the relative valence (positive and negative) of others’ scores and measured how exposures to higher and lower scores affect the focal evaluator’s propensity to change the initial score. We found causal evidence of a negativity bias, where evaluators lower their scores by more points after seeing scores more critical than their own than raise them after seeing more favorable scores. Qualitative coding of the evaluators’ justifications for score changes reveal that exposures to lower scores were associated greater attention to uncovering weaknesses, whereas exposures to neutral or higher scores with increased emphasis on non-evaluation criteria, such as confidence in one’s judgment. The greater power of negative information suggests that information sharing among expert evaluators can lead to more conservative allocation decisions that favor protecting against failure than maximizing success.

The Effects of Prize Structures on Innovative Performance by Joshua Graff Zivin; Elizabeth Lyons

AER Papers and Proceedings

Successful innovation is essential for the survival and growth of organizations, but how best to incentivize innovation is poorly understood. We compare how two common incentive schemes affect innovative performance in a field experiment run in partnership with a large life sciences company. We find that a winner-takes-all compensation scheme generates significantly more novel innovation relative to a compensation scheme that offers the same total compensation but shares it across the ten best innovations. Moreover, the winner-takes-all scheme does not reduce innovative output on average and, among teams of innovators, generates more output than the less risky prize structure.

The Janus face of artificial intelligence feedback: Deployment versus disclosure effects on employee performance by Siliang Tong, Nan Jia, Xueming Luo, Zheng Fang

Strategic Management Journal

Companies are increasingly using artificial intelligence (AI) to provide performance feedback to employees, by tracking employee behavior at work, automating performance evaluations, and recommending job improvements. However, this application of AI has provoked much debate. On the one hand, powerful AI data analytics increase the quality of feedback, which may enhance employee productivity (“deployment effect”). On the other hand, employees may develop a negative perception of AI feedback once it is disclosed to them, thus harming their productivity (“disclosure effect”). We examine these two effects theoretically and test them empirically using data from a field experiment. We find strong evidence that both effects coexist, and that the adverse disclosure effect is mitigated by employees' tenure in the firm. These findings offer pivotal implications for management theory, practice, and public policies.

Congruence Between Leadership Gender and Organizational Claims Affects the Gender Composition of the Applicant Pool: Field Experimental Evidence by Mabel Abraham, Vanessa Burbano

Organization Science

The extent to which men and women sort into different jobs and organizations—namely, gender differences in supply-side labor market processes—is a key determinant of workplace gender composition. This study draws on theories of congruence to uncover a unique organization-level driver of gender differences in job seekers’ behavior. We first argue and show that congruence between leadership gender and organizational claims is a key mechanism that drives job seekers’ interest. Specifically, many organizational claims are gender-typed, such that social claims activate the female stereotype, whereas business claims activate the male stereotype. Thus, whereas female-led organizations making social claims are gender-congruent, male-led firms making the same claims are gender-incongruent. Beyond demonstrating a general preference among job seekers for congruence, we also find that female job seekers are most interested in working for organizations that are simultaneously congruent and provide credible signals that they are fair and equitable employers. The congruence of leadership gender and organizational claims thus affects the gender composition of applicant pools for otherwise identical jobs.

Mitigating Gig and Remote Worker Misconduct: Evidence from a Real Effort Experiment by Vanessa C. Burbano and Bennett Chiles

Organization Science

Employee misconduct is costly to organizations and has the potential to be even more common in gig and remote work contexts, where workers are physically distant from their employers. There is thus a need for scholars to better understand what employers can do to mitigate misconduct in these non-traditional work environments, particularly as the prevalence of such work environments is increasing. We combine an agency perspective with a behavioral relationship-based perspective to consider two avenues through which gig employers can potentially mitigate misconduct: 1) through the communication of organizational values and 2) through the credible threat of monitoring. We implement a real effort experiment in a gig work context that enables us to cleanly observe misconduct. Consistent with our theory, we present causal evidence that communication of organizational values, both externally-facing in the form of social/environmental responsibility and internally-facing in the form of an employee ethics code, decreases misconduct. This effect, however, is largely negated when workers are informed that they are being monitored. We provide suggestive evidence that this crowding-out is due to a decrease in perceived trust that results from the threat of monitoring. Our results have important theoretical implications for research on employee misconduct, and shed light on the tradeoffs associated with various potential policy solutions.

The Demotivating Effects of Communicating a Social Political Stance: Field Experimental Evidence from an online Labor Market Platform by Vanessa C. Burbano

Management Science

Despite a recent surge in corporate activism, with firm leaders communicating about social-political issues unrelated to their core businesses, we know little about its strategic implications. This paper examines the effect of an employer communicating a stance about a social-political issue on employee motivation, using a two-phase, preregistered field experiment in an online labor market platform. Results demonstrate an asymmetric treatment effect of taking a stance depending on whether the employee agrees or disagrees with that stance. Namely, I observe a demotivating effect of taking a stance on a social-political issue with which employees disagree and no statistically significant motivating effect of taking a stance on a social-political issue with which employees agree. This study has important implications for the nascent scholarship on corporate activism, as well as the scholarship on strategic human capital management.

Delegation in Hiring: Evidence from a Two-Sided Audit by Bo Cowgill; Patryk Perkowski

Working Paper

Firms increasingly delegate job screening to third parties. Outsourced recruiters introduce agency issues that can distort worker/employer matching. We present a simple model illustrating the tension between recruiters' use of private information about candidate preferences, and reputational incentives to impress employers. We then execute a novel two-sided audit experiment by directly engaging recruiting intermediaries. We show that agents' career concerns strongly shape selection decisions. Recruiters over-interview employer-favored candidates unlikely to reciprocate interest, while under-select candidates with stronger mutual interest. By changing outside offers, recruiters' career concerns affect candidates' negotiating leverage. We conclude by discussing extensions of our audit methodology.


Copyright © 2021 Nesta, All rights reserved.

Our mailing address is:
58 Victoria Embankment, London, England EC4Y 0DS

Forward to a friend

Subscribe to this newsletter

Want to change how you receive these emails?
You can update your preferences or unsubscribe.