A maturity model for software startup ecosystems

Abstract

Resulting from the digital revolution of the last decades, multiple startup hubs flourished across the globe in the past 10 years. Healthy environments for the development of innovative, nascent digital enterprises require a well-balanced variety of agents and supporting processes, which we collectively call a software startup ecosystem. These ecosystems are fundamental for the insertion of countries in the digital economy of the twenty-first century. However, having all the elements that compose such environments in the most advanced and prolific state is difficult and relatively rare.

In this paper, we show that startup ecosystems can evolve over time passing through a sequence of maturity level stages. For that, we introduce a maturity model for software startup ecosystems based on systematic qualitative research around a multiple case study we conducted across three ecosystems. The study was carried out over 4 years and included an extensive array of data collection mechanisms such as literature reviews, expert interviews, and observations in three relevant ecosystems (Tel-Aviv, São Paulo, and New York); all collected data were analyzed with techniques based on Grounded Theory, resulting in a conceptual framework of software startup ecosystems.

Finally, we developed a maturity model for startup ecosystems, which helps us understand their evolution and dynamics. Moreover, it can serve as a basis for stakeholders in less mature ecosystems to analyze their environment, identify weak spots, and propose policies and practical actions for improving their ecosystems over time.

Background

In the last two decades, we observed the rise and maturation of many software startup ecosystems around the world. The technological revolution has driven society evolution, prompted by broader access to the Internet and the popularization of mobile devices; likewise, society’s progress drives technological evolution in a co-embedded evolution phenomenon. The Global Entrepreneurship Monitor, a long-term study conducted by a consortium of universities, shows that human capital and social capital co-evolve (Reynolds et al. 2000; Singer et al. 2015). Given the hundreds of technological clusters present in different countries, it is difficult to identify each ecosystem’s level of development. This paper proposes a methodology to measure such maturity with respect to multiple factors, enabling the ability not only to compare different ecosystems, but, more importantly, to propose practical actions that can lead to meaningful improvements in existing ecosystems.

As Daniel Isenberg argues, “There’s no exact formula for creating an entrepreneurial economy; there are only practical, if imperfect, road maps.” Instead of aiming to imitate successful ecosystems, each region should identify and develop its own qualities (Isenberg 2010). Isenberg also proposes a conceptual model for entrepreneurship ecosystems, which maps different agents in the ecosystem and proposes that they must work together. The entrepreneurship ecosystem can be viewed as a new paradigm for economic policies (Isenberg 2011). Isenberg’s model is based on the OECD entrepreneurial determinants, which proposes indicators for measuring an ecosystem’s performance in six areas: regulatory framework, market conditions, access to finance, creation and diffusion of knowledge, entrepreneurial capabilities, and entrepreneurship culture. A limitation of this model is that it misses ecosystems’ dynamics and the connectivity aspects.

Economic theory shows that entrepreneurs are the prime forces in modern economic development. Significant changes of economic systems are impossible without them (Schumpeter 1934). Besides creating new jobs and generating wealth in society, entrepreneurs and their startups foster the technological innovation in industries. New venture creation is statistically linked to both job creation (Acs and Armington 2004; Endeavor Brasil 2015) and regional development (Kasturi and Subrahmanya 2014). High rates of entrepreneurial activity are strongly related to the growth of local economies. Entrepreneurial market activity is mostly a decentralized and unplanned process (Lewin 2011; Koppl 2008), in which innovative companies must effectively interact with each other to achieve success (Olsson and Bosch 2015); thus, technological entrepreneurs act in the context of complex entrepreneurship ecosystems, which can be viewed as a new paradigm for economic policies (Isenberg 2011).

In our research, we focused on technology entrepreneurs and their software startups: companies with a potential for high-growth and scalable business models (Blank and Dorf 2012). Startups usually have to pivot their strategy, especially in the first 2 years, until they find their product-market fit (Terho et al. 2015). A supportive startup ecosystem can help entrepreneurs during this unstable period. We define a startup ecosystem as a limited region, roughly within a 50-km (or 1-h travel) range, formed by people, their startups, and various types of supporting organizations, interacting as a complex system to create new startup companies and evolve existing ones.

Porter introduced the concept of clusters in 1990 (Porter 2011), as a geographically close group of interconnected companies and associated institutions in a particular field. Differently from the cluster concept, which can be viewed as a static asset, ecosystems are dynamic complex structures in which stakeholders co-evolve (Moore 1993) based on both competition and cooperation (Peltoniemi 2004). Thus, clusters are components within ecosystems. Besides, we define a startup ecosystem as a limited geographic region, the boundaries of these regions are not perfectly clear, and the ecosystem is not dependent on these borders to exist. Boundaries are useful specially for the purpose of defining limits between different ecosystems that are geographically close. Therefore, for example, San Francisco and San Jose are far enough to be considered two different ecosystems, but at the same time, they are close enough to form a single larger ecosystem known as the Silicon Valley. Similarly, Tel Aviv, Haifa, and Jerusalem are three distinct ecosystems, all of them within the broad Israeli ecosystem.

Any healthy entrepreneurial ecosystem directly impacts on entrepreneurs’ lives (Jayshree and Ramraj 2012). Several studies try to identify gaps in innovation ecosystems and propose practical actions to improve their performance, with examples in Germany (Voss and Müller 2009; Sternberg 2013), India (Kasturi and Subrahmanya 2014), Portugal (Vaz et al. 2014), and Israel (Kon et al. 2015). Some industry initiatives such as the Startup Genome¹ try to map the characteristics of all startup ecosystems around the globe.

This paper is based on a multiple case study (Stake 2013) applying rigorous qualitative methods (Corbin and Strauss 2007; Stol et al. 2016; Maxwell 2012) and carried out in three different software startup ecosystems: Tel Aviv (Israel), São Paulo (Brazil), and New York (USA). Rather than only mapping the innovation ecosystems’ characteristics and proposing actions and policies for those locations (Frenkel and Maital 2014), the primary objective of this research was to understand their dynamics and explore how they evolved over time. Understanding each ecosystem’s characteristics as a snapshot in time is very important, but evaluating their dynamics allowed us to understand the path that ecosystems followed to grow in a sustainable way. By mapping the road, we can show to ecosystem stakeholders the next steps they need to take to advance in the evolutionary process.

The general objective of this research was to advance the understanding of how software startups work, what are the elements that influence their behavior, and how startups relate with other players in their ecosystem. From this general objective, we derived our specific objectives: (1) achieving a better comprehension of existing startup ecosystems, with the development of a generic conceptual framework of software startup ecosystems; (2) instantiating the conceptual framework at, at least, three different ecosystems, in three different regions of the world, analyzing their characteristics, strengths, and weaknesses; (3) developing a methodology to compare multiple ecosystems, highlighting their similarities and differences; and (4) creating a model to map ecosystem evolution and dynamics.

The literature includes many articles and books about general entrepreneurship, but very few works that focus on software startups and the ecosystems that produce them. After an in-depth analysis of two ecosystems (Tel Aviv and São Paulo), we released the first version of a maturity model for software startup ecosystems (Cukier et al. 2015b; 2015a) to represent this evolutionary process. We then received a substantial amount of feedback from practitioners and researchers both in research workshops and in interviews with experts. Based on this feedback, we ran a third iteration of the research in New York to validate the final version of the model, which we present here.

After the refinement process, the final model, which includes information collected from over 100 experts from the three countries, ended up considering 21 evaluation factors, such as access to global markets, mentoring quality, accelerator’s quality, human capital, and entrepreneurship in universities. It classified ecosystems in four levels of maturity: nascent (M1), evolving (M2), mature (M3), and self-sustainable (M4).

In our last iteration for validation with the multiple case study, we found that the New York City startup ecosystem fits perfectly in the final model (Cukier et al. 2016). In less than 15 years, this ecosystem evolved from the bottom level of maturity (nascent/evolving) to the top level (mature/self-sustainable). This case shows not only that it is possible for a particular region to develop a healthy entrepreneurial environment, but also that this development progresses through a path of multiple phases, in which each phase can be determined by different characteristics and requires specific management approaches. Moreover, this particular evolution is closely related to the moment when technology invaded mainstream businesses and when traditional business centers started to become technology centers.

The results of this 4-year research were disseminated as technical reports (Kon et al. 2014; 2015; Cukier et al. 2016) and a PhD thesis (Cukier 2017); the complete, final result of our research is published for the first time in this paper. It details the multiple case study and how it can be used to generalize a theory about startup ecosystems. The “Related work” section discusses existing literature and theory about startup ecosystems. The “Methods” section explains the methodology we used to collect data and analyze the results. The “Results and discussion” section presents our findings. Finally, “The startup ecosystem maturity model—final version” section states our conclusions, suggestions for future work, and the research limitations and threats to validity.

Related work

Startup ecosystems cannot be analyzed as static entities. Similar to biological ecosystems, they behave like living organisms and change over time. Some changes are planned or somehow controlled, while others are results from unexpected forces acting within and outside the ecosystem.

The groundwork for the startup ecosystem literature was laid years before both the terms “ecosystem” and “startup” began to be broadly used and understood in the context of company creation. Dating back to the 1980s and 1990s, scholars have studied geographic regions around the world where entrepreneurs have successfully emerged, seeking to understand the reasons behind that success (Feldman 1994; Rogers and Larsen 1984; Saxenian 1994; Malecki 1997). Others have focused on prescriptions for supportive environments for emerging businesses that feature human development and other services (Bennett and McCoshan 1993; Johannisson 1993). This literature made early contributions to the idea of a context supportive to both entrepreneurs and their enterprises, which was later fleshed out in the model of a “pipeline of entrepreneurs and enterprises” for managing a community’s portfolio of businesses (Lichtenstein and Lyons 2001).

In 2001, Lichtenstein and Lyons presented their “entrepreneurial development system,” which focused on the development of entrepreneurial skills through a community-wide or regional coaching and support system, as the primary strategy for creating wealth and economic prosperity (Lichtenstein and Lyons 2001).

The theme “software startup” is not so new, even if there is still a broad research agenda for it (Unterkalmsteiner et al. 2016). Software startup ecosystems are a novel object of study, although we already have ample examples demonstrating that these ecosystems pass through different phases during their development and that they can eventually degrade or die, as has been reported in Atlanta (Breznitz and Taylor 2014).

The term “startup ecosystem” appeared in the literature around 2005 and, according to Google Scholar, the occurrence of this term grew exponentially from 2010 to 2016, as depicted in Fig. 1. In fact, the notion of startup ecosystems emerged when technology (especially the Internet, and later, mobile systems) entered the mainstream and became a crucial aspect for innovation, transforming many traditional business centers into technology centers.

Fig. 1

Papers containing the term “startup ecosystem.” Source: Google Scholar

Nevertheless, very few works explore the ecosystem’s dynamics and evolutionary nature. To analyze them, a snapshot from a given point in time is not enough (Mason and Brown 2014). Ecosystems must be evaluated over the course of years, in particular, a longer period over which we can observe the ecosystem maturation process.

As many previous studies claim, culture is a profoundly significant aspect that defines the ecosystem’s characteristics. Thus, deciphering its culture is one way to understand an ecosystem. Hofstede presented measurements for cultural aspects in many different countries (Hofstede et al. 2010). Places with significantly different cultural behaviors can have their own successful ecosystems, which shows that specific cultural characteristics themselves are not a requirement for the existence of healthy ecosystems, but rather a base over which ecosystems evolve. Every region or country has a different entrepreneurial identity, which had in part been attributed to culture (Krueger et al. 2013). Besides culture, the success of an innovation ecosystem highly depends on the level of inter-connectivity between its players (Iansiti and Levien 2004). Connectivity matters and improves as an ecosystem progresses (Stephenson 2008).

Stam developed a model to measure ecosystems (Stam 2018) based on ten called elements: formal institutions, entrepreneurship culture, physical infrastructure, demand, networks, leadership, talent, finance, new knowledge, and intermediate services. This study was based on a case study of 12 cities only in Netherlands and defines an index for ecosystem evolution.

Lemos mapped entrepreneurship ecosystems based on the perspective of a research university (Lemos 2011; 2012). His model has elements similar to Isenberg’s, but primarily focuses on the research university elements, and not on the startup ecosystem as a whole. First-class universities all around the world play a vital role in the development of the entrepreneurship ecosystem around them (Sternberg 2013), for example: Stanford in Silicon Valley (Piscione 2013), Technion in Israel (Kon et al. 2014), and Cornell in New York (Cometto and Piol 2013). Similarly, in São Paulo, USP serves as an unofficial hub for much of the entrepreneurial activity in the city and has a number of entrepreneurship initiatives on campus, including specific courses, an incubator, and student entrepreneurship groups. USP was also considered, in a recent ranking, the best entrepreneurial university in Brazil (Neves and Rosso 2016). While research shows that the university plays a crucial role on the development of a healthy ecosystem, the presence of a high-quality university is only one of many factors that characterize an innovation hub, as we can see in a report by Endeavor comparing the level of entrepreneurship in different Brazilian cities (Endeavor Brasil 2015).

The World Economic Forum mapped eight pillars of entrepreneurial ecosystems (Foster et al. 2013), namely: (1) accessible markets, (2) human capital workforce, (3) funding and finance, (4) mentors and advisors support system, (5) regulatory framework and infrastructure, (6) education and training, (7) major universities as catalysts, and (8) cultural support. All these eight elements are present in our proposed maturity model and conceptual framework. Our model goes further by not only mapping the ecosystem pillars but also exploring the relationships and inter-dependencies among them.

Stangler and Bell-Masterson propose four indicators of entrepreneurial ecosystem vibrancy: density, fluidity, connectivity, and diversity (Bell-Masterson and Stangler 2015). Although the authors emphasize the importance of the dynamics when analyzing ecosystems, they do not propose a practical method for evaluating the ecosystem maturity level.

The triple helix model proposes that innovation ecosystems can be managed in a top-down approach from three perspectives: government, universities, and industries (Etzkowitz and Leydesdorff 2000). Brännback et al. challenge this model by arguing that it fails to include the entrepreneur as the most important agent in the entrepreneurship ecosystem and that the bottom-up approach is more effective (Brannback et al. 2008). Brad Feld’s model also emphasizes the importance of the entrepreneur; he presents the “Boulder Hypothesis” (Feld 2012) attributing four essential characteristics to a successful startup community: (1) it must be led by entrepreneurs and not by other important players, such as government, universities, service providers, and big companies, which Feld calls “feeders”; (2) the leaders (entrepreneurs) must have a long-term commitment to the community (at least 20 years); (3) it must be inclusive, which means that everyone who wants to participate must be welcome; and (4) it must have high-quality events to engage people, especially acceleration programs and mentoring sessions. Less fragmented ecosystems would score higher on all four elements. Recent studies show that policies that focus on bottom-up approaches are more efficient when developing startup ecosystems (Stam 2015), identifying the entrepreneur as the primary change agent, while the traditional triple helix model tends to discard the entrepreneurs to focus only on governments, universities, and industries.

Changes in ecosystems are observed over time, and some differences can take years or even decades to be observed. Ecosystems are dynamic and evolutionary per se, rather than a static phenomenon that can be captured by a snapshot at a given point in time (Mason and Brown 2014). The Startup Ecosystem Report from 2012 (Herrmann et al. 2012) proposes a ranking of the top 20 ecosystems in the technology economy. It puts Silicon Valley as a benchmark and compares other ecosystems to it. Three years later, in another report by the same institution, the Global Startup Ecosystem Ranking for 2015 (Herrmann et al. 2015) revises the 2012 version, presenting a new landscape of ecosystems that shows new technological hubs entering the ranking, as well as old startup agglomerations that did not evolve enough to enter in the new top 20. The latest version, the Global Startup Ecosystem Report 2017, presents a broader perspective, adding several ecosystems from Asia, including Beijing in the fourth place (Startup Genome 2017). Questions that arise include the following: what happened to those ecosystems that fell out of the ranking? What did the ecosystems that entered in the ranking do to scale up? Does ranking higher mean improving, and lower mean worsening? Could the evolution across maturity level stages be evidence of a virtuous cycle (Björklund and Krueger 2015)?

This work was built on top of the previous presented research, going further on (1) mapping the relationships between ecosystem agents, (2) analyzing not only the static characteristics of ecosystems but also their dynamics, (3) proposing a practical methodology for classifying ecosystem maturity level, and (4) mapping the critical factors of each maturity level as well as the path to the next level.

Methods

This research was performed in three phases, each phase serving as the basis for the next one as depicted in the pyramid in Fig. 2. The objective of phase 1, at the base of the pyramid, was to map the components and factors that characterize software startup ecosystems as well as the relationships among them. This phase was based on Grounded Theory techniques (Corbin and Strauss 2007).

Fig. 2

Maturity model research phases

In phase 2, the objective was to validate and refine the map produced in phase 1, as well as propose a maturity model for the evolutionary process that ecosystems undergo. During this phase, the method mixed literature review, expert workshops, and focus groups.

Phase 3 aimed to refine and validate the maturity model proposed in phase 2. This last phase completes our multiple case study. A case study is a research method focused on understanding the dynamics of research objects (Runeson and Höst 2009; Yin 2013). Multiple case design is used to analyze complex research entities, enabling the induction of rich and reliable models (Stake 2013; Yin 2013). It consists in analyzing multiple instances of the research object, which were startup ecosystems in our case. The next subsections explain in detail each research phase.

Phase 1: Startup ecosystem conceptual framework

In the first phase, we used qualitative methods with elements from the Grounded Theory (GT) (Corbin and Strauss 2007) to identify the key factors that led to the emergence of a successful ecosystem. GT is a complex method fundamentally different from the traditional hypothetico-deductive research model. Grounded Theory-based studies have been growing in the Computer Science field in the last decade. The goal of GT is to generate a theory other than testing or validating an existing one (Stol et al. 2016). GT proposes an iterative method of collecting-coding-analyzing data. Each iteration brings new insights and ideas. This new ideas are applied to the new iteration, changing the mechanisms of collecting-coding-analyzing. This process repeats until we reach a theoretical saturation point, when all concepts, properties, and relationships are already mapped and new iterations do not bring new elements to the model.

Our goal was to develop a conceptual framework (Miles and Huberman 1994) of the software startup scene that could help analyze the current status of ecosystems, as well as locate opportunities for their improvement. At that time, we did not delve deeply into related work on frameworks and models of startup ecosystems, as we did not want to be biased in advance; rather, we wanted our framework to emerge from the data collected on the field. This approach is described by Corbin and Strauss (Corbin and Strauss 2007) as a particular use of theoretical frameworks in qualitative studies: the researcher first develops a light theoretical body that provides a useful list of concepts, insights, and direction, allowing them to remain open to new concepts and ideas that emerge from field data as they carry out the study. We followed the GT approach that recommends to limit the exposure to the existing literature (Stol et al. 2016), preventing the researcher from testing existing theory rather than being open-minded to new discoveries. We also followed other principles of the GT method:

• Treat everything as data, not only the formal interviews, but also the informal interactions

• Immediate and continuous data analysis during all data collection period

• Theoretical sampling by identifying new sources of data based on the continuous analysis

• Theoretical sensitivity by identifying the relationships between startup ecosystem elements

• Memoing by creating notes and recordings of all data collected to be consulted later

• Constant comparison between the raw collected data and the analyzed data and categorization

• Memo sorting by oscillating between the memos and the emerging theory

The objective of this first phase was to answer the following research questions²:

• RQ1—What are the sociocultural characteristics found in startup ecosystems that foster the entrepreneurial spirit and what are the institutional mechanisms that promote entrepreneurship?

• RQ2—What is the role of education in fostering entrepreneurship in startup ecosystems? What are the formal and informal, explicit and implicit pedagogical materials and mechanisms that nurtures the entrepreneurial spirit?

• RQ3—What are the characteristics of successful innovative teams and entrepreneurs? What is the prime motivation of the software entrepreneur?

• RQ4—Which technological and methodological aspects influence the success of software startups, and how? In particular, what is the role played by languages, frameworks, patterns, and models? What is the role of open source software?

• RQ5—What opportunities exist for further development of the startup ecosystem? What could be a threat?

We began the first case study in Israel. To collect the data, we used meetings with experts, semi-structured interviews, observations, and a questionnaire. We have long experience in software development (15 and 25 years), both in the academia and in the industry. We brought to the research a comprehensive knowledge of the focused field (software) and also an outsider, non-Israeli perspective, which allows us to see characteristics and facts sometimes not grasped by researchers who have been immersed in the culture for several decades. We partnered with Israeli researchers (Kon et al. 2014) who specialized in qualitative methods and software engineering education and entrepreneurship; they brought to the team a more in-depth knowledge of the local culture as well as complementary skills relevant to the research.

To select the people to interview and startups to visit, we began collecting suggestions from staff and faculty members from the Technion Bronica Entrepreneurship Center, from the Hebrew University of Jerusalem Yissum Technology Transfer company, and from the economic news website of the Israeli Ha’aretz newspaper, The Marker. We then followed a snowball approach, in which people that we met and interviewed recommended additional contacts. Overall, we approached 78 people via email or Linkedin messages and were able to meet, in person, 48 of them.

From August to December, 2013, Fabio Kon carried out 48 meetings in several cities, primarily in Tel Aviv, followed by Haifa and Jerusalem. Fourteen of these comprised informal conversations with experts in the high-tech and startup industry, on which the researcher took detailed notes. The core of the material was composed of additional 34 semi-structured interviews, whose full audio was recorded in 25 cases, and with detailed written notes taken in the 9 remaining cases. Most formal interviews lasted for about 1 h; the shortest of all lasted for 15 min, while the longest lasted for 2 h and 16 min. These interviews mostly covered startup founders, CEOs, and CTOs, but also a few angel investors, venture capitalists (VCs), developers, and incubator and accelerator managers. During this process, 25 different startups and 6 accelerator/incubators were visited and observations were written down. Fabio Kon also attended, writing systematic notes, several events, lectures, seminars, and meet-ups, which are characteristic of the Israeli startup ecosystem.

As the interviews were carried out, the critical elements of the startup ecosystem, as mentioned by the interviewees, were annotated and the conceptual framework (Miles and Huberman 1994) was iteratively constructed and refined by the two authors of this paper. To answer RQ5, a selected group of the 25 most experienced participants was asked to answer a SWOT questionnaire in an on-line form³; 20 of them provided their input.

We created a protocol describing the full interaction process with interviewees. The protocol describes how to approach the contacts, as well as the interview process and follow-up. A first version of this protocol was used for interviews in Israel. Later, in the second (the “Phase 2: Conceptual model validation, refinements, and maturity model proposal” section) and third (the “Phase 3: Startup ecosystem maturity model validation” section) phases of the research, we refined the protocol, producing new versions for the interviews in São Paulo and New York City⁴.

As qualitative studies can be highly context-and-case-dependent, we observed four principles to promote a trustworthy study: credibility, transferability, dependability, and confirmability (Lincoln and Guba 1985). To achieve credibility, we first developed the data collection instruments from practitioners and experts’ opinion, consulting with highly experienced professionals in innovation ecosystems and software development. Although we have used a purposive sampling of interviewees from top universities and startups, combined with a snowball approach, we tried to include participants by considering similarities, dissimilarities, redundancies, and varieties to acquire a greater knowledge of the wider group. We collected and analyzed data in a systematic and iterative way, from a high number of participants, which improved both confirmability and dependability. To promote transferability, we described protocol details, the data collection and analysis process, and, finally, included quotations with our major findings in the technical reports we produced.

Phase 2: Conceptual model validation, refinements, and maturity model proposal

We performed an extensive review of the related literature after the first case study in Israel (Kon et al. 2015). Article sources included Google Scholar, recommendations from ecosystem research experts, and snowballing from references within the articles we read. The keywords used in the search were “startup ecosystem” and “entrepreneurship ecosystem”. Though not a fully comprehensive systematic review, it encompassed 248 readings, including books (45), reports (17), theses (4), and articles (182) published on conferences and academic journals on business, high-tech ventures, entrepreneurship, regional development, innovation, software, and management. The literature findings most relevant to our work were presented in the “Related work” section.

After the literature review, we initiated a second case study in the São Paulo ecosystem, which was reported by a member of our research group as a Masters thesis (Santos 2015). A new round of interviews were performed between May 2014 and January 2015. We conducted 41 meetings with 32 startups, 3 accelerators, 4 venture capitalists, 1 angel investor, and 1 government agent. Again, the data collection and analysis followed principles of the Grounded Theory approach and the conceptual model was refined. The list of interviewees was created using a process similar to the Israeli case study. The first author (Daniel) has a long professional experience in the São Paulo software industry. He worked as CTO in a growth-stage startup that received investment from renowned venture capital groups from Brazil and Silicon Valley, and he later founded a software startup and raised both angel and public investment. Therefore, he already had important connections with industry practitioners (invested startups, venture capital groups, and accelerator managers). The second author (Fabio), as a full professor in one of the most respected universities in Brazil, also brought a list of both industry and academy insiders in the startup ecosystem.

The study of this particular ecosystem allowed us to identify new factors and characteristics that were neglected in the Israeli study. Table 1 presents some differences between the São Paulo and Tel Aviv ecosystems. Although similarities exist in the two ecosystems, we emphasized the differences to show how collecting data from both enriches our model by making it more generalizable.

Table 1 Main differences across the Tel-Aviv, São Paulo, and New York startup ecosystems

Whereas we applied a SWOT analysis to find opportunities for further development in the Israeli ecosystem, in São Paulo, we used a different approach. We applied a qualitative technique based on a systematic workshop/focus group that we executed in São Paulo (Cukier et al. 2016) following a methodology proposed by Frenkel and Maital. The inputs for the innovation ecosystem map are based on a structured collaborative discussion (experts’ workshop) conducted among experts from various realms and disciplines in São Paulo participating in this collaborative exercise. The methodology we used for mapping the national innovation ecosystem was developed and implemented in an earlier large-scale study (Frenkel and Maital 2014). The objective of the experts’ workshop was to identify fundamental “anchors” and “processes” that comprise the main elements of the innovation ecosystem around São Paulo’s largest research university. We define these two key concepts as such:

• “Quality anchors” are ecosystem’s strengths, or core competencies, on which innovation can be built, such as a high level of human capital, or the existence of strong world-class scientific and technological infrastructure.

• “Processes and trends” are processes, such as vocational training programs, tax incentives, and R&D funding, that can enable countries or regions to overcome strategic innovation weaknesses, or constraints that hamper innovative initiatives and policies.

The identification of the anchors and processes was carried out in the experts workshop, which took place in the University of São Paulo main campus on August 22, 2014. To compile these results, presented in detail in (Cukier et al. 2016), we put together a group of two local experts from the São Paulo ecosystem and two innovation scholars from abroad, with no previous knowledge of this local environment. Our goal was to achieve a good balance between insiders’ and outsiders’ views of the São Paulo culture and processes around innovation.

Following the experts workshop, we published a technical report describing the results. Up to that moment, we had three main outputs: (1) insights and practical actions regarding the improvement of the São Paulo startup ecosystem, (2) data to complete our generalized conceptual framework of startup ecosystems, and (3) a strong recommendation from some experts to focus our analysis not only on static characteristics of ecosystems, but also on their dynamics. Based on that, we began to developed a maturity model for startup ecosystems.

The maturity model was built iteratively. Since we already had a consistent list of the elements in an ecosystem (startup, entrepreneur, funding bodies, etc.), we sought to identify consistent evaluation metrics for each element.

Initially, after long discussions among the core team of researchers involved in this effort, we decided to define, for each metric, an evolution scale in three levels (L1, L2, and L3). Each level had a value or a value range for each metric. For example, the specialized media players metric had values < 3 for L1, values 3 to 5 for L2, and values > 5 for L3. These initial values were not completely arbitrary. For each metric, there was a reasonable valuation, trying to match the existing ecosystems we already analyzed, and also considering ecosystem rankings we found in the literature. In the “Results and discussion” section, we describe the complete version of the model in detail.

After exposing and discussing this initial version of the model with multiple experts from different countries, we not only refined the model by adjusting the metric range values but we also concluded that we needed more data to further refine and validate the model. We decided to do a third case study, adding a set of questions to the research protocol⁹, now focusing on the discovery of the evolution process. The next phase was executed to fine-tune and validate the maturity model we had created.

Phase 3: Startup ecosystem maturity model validation

During the final phase or four research, we ran a case study in the New York City startup ecosystem. The main focus was not to identify how ecosystems operate or the agents involved (this was already accomplished in phases 1 and 2), but rather to clarify how ecosystems evolve and mature over time and validate the maturity model proposed in the end of phase 2.

This part of the research was conducted in the New York City region, in a range of 15 miles from the Manhattan center, on October 2015. The qualitative research method included performing 25 semi-structured interviews with NYC startup ecosystem agents among entrepreneurs (14), investors (4), scholars (4), and other supporting players (3). The interviewees were selected by snowballing (Goodman 1961) our network in both academia and industry. Only one participant was less than 30 years old; the average age was 42 with standard deviation of 11. They were 17 males and 8 females in roles including CEO, COO, CTO, lawyer, professor, manager, founding partner, and writer. All interviewees had at minimum an undergraduate degree: 38% had a master’s or MBA, and 13% were PhDs.

The interview protocol was a refined and updated version of the one used in our previous research in both Israel and São Paulo. The protocol was designed to answer the following research questions:

• A. What are the minimum requirements for a startup ecosystem to exist in its nascent stage?

• B. What are the requirements for a startup ecosystem to exist as a mature self-sustainable ecosystem?

• C. What are the stages that ecosystems pass through? Can they regress or die?

• D. Can people proactively interfere in the evolution of ecosystems? Is it possible to develop ecosystems into mainstream ecosystems such as Silicon Valley, generating tens of high growth global startups? How many of these could exist in the world?

To reinforce the qualitative insights from the interviews, we explored quantitative data about startups from the Crunchbase database. Despite the fact that this database is not an official source of information about all existing startups, it is a good representation of the reality. Most relevant startups, especially those that received or want to receive investment, are listed in this database. Moreover, absolute and precisely correct numbers are not necessary for the conclusions we present.

One of the reasons we chose New York was because our literature review showed that this ecosystem experienced a tremendously fast evolution over the last two decades. Another reason we chose New York was that it is an ecosystem with different characteristics compared to the two ecosystems we analyzed before (see Table 1). Thus, by examining it, we could validate whether our maturity model proposal adhered to three different realities, making it more robust and generalizable, what is normally referred to as triangulation in qualitative research. Differently from statistical generalization, the analytic generalization is not defined by population that has been sampled, but to a theory of the phenomenon being studied, a theory that may have much wider applicability than the particular case studied (Gibbert et al. 2008).

During this phase, we also collaborated with JF Gauthier, a specialist from Startup Genome, a company that focuses on helping startups measure their success based on standard metrics. JF was in charge of creating a similar model to evaluate the ecosystem evolution process, similar to our proposal. The Compass model (Compass 2015) is significantly different, since it is simpler and has fewer metrics. Nevertheless, the inputs from Gauthier were valuable in our process of refining our model.

Besides JF, we also interviewed other specialists from Tel Aviv and São Paulo, showing them the model, at first hiding the metrics values and systematically asking them to give us their own opinion about what value should be in each level for each metric. During these discussions, we adapted the values and also decided to completely remove some metrics that were not well received by a significant number of interviewees.

After these discussions and updates, a refined version of the model was presented in an international Workshop (Cukier et al. 2015a) promoted by the Software Startups Global Research Network¹⁰. During the workshop, the model was praised, but also received constructive criticism in some aspects. We ran a few more iterations changing the existing model, considering the feedback we received from the specialists during the workshop, and also the new insights from the New York ecosystem case study. We then presented a newer version of the model in a subsequent workshop (Cukier et al. 2016). Based on another round of feedback from the specialists, we created the last version of the model, which we now present in the next chapter (in the “Related work” section), after presenting the results from all three phases of this research.

Results and discussion

In this section, we discuss the results from the three phases of our research: the startup ecosystem conceptual framework, insights from the São Paulo Ecosystem, and the New York Case-study Adherence to the Maturity Model.

Phase 1 results: startup ecosystem conceptual framework

This first phase of the research resulted in the conceptual map of startup ecosystem key participants as well as the relationships among them. The resulting map was first published as a technical report in 2014 (Kon et al. 2014). Figure 3 depicts the final version of the model, after the refinements of research phase 2. Since the figure is complex, we suggest examining it like a traveler looks at a map, navigating through it and not at first attempting to understand all details. While it may take some time to understand the whole topology, the map clearly shows that the elements that play a role in a startup ecosystem are numerous and that there are a multitude of relationships among them.

Fig. 3

Startup ecosystem conceptual framework

We can briefly describe the framework in one paragraph as follows. “An entrepreneur creates a startup by identifying opportunities in the market. Startups face multiple challenges to achieving market fit (Giardino et al. 2015) and becoming successful. For this reason, the entrepreneur seeks support from family, friends, and other personal connections, who are part of a society and culture that influence the entrepreneur’s behavior. Demographics characteristics such as language, race, religion, and gender influence the culture and creates opportunities and barriers to the entrepreneur. Geopolitical status also influences the culture and creates opportunities and barriers for the startup. Universities and research centers provide knowledge in technologies that enable the startup, they also prepare the entrepreneur and offer networking possibilities. Universities and research centers also collaborate (or compete) with entrepreneurs in the technology transfer process (Berbegal-Mirabent et al. 2012). Successful, experienced entrepreneurs serve as mentors to novices. Universities and established companies run incubators and accelerators that train and instrument the startup with methodologies such as agile methods (Abrahamsson 2002), lean startup (Ries 2011), customer development (Blank 2013), and disciplined entrepreneurship (Aulet 2013). Eventually, established companies buy, compete, or collaborate with the startup. Private funding bodies like angel and venture capitalists mentor and invest on startups, which can also get financial resources from governmental programs through R&D funding agencies or tax incentives. The existing legal framework (labor laws, tax laws, IP, patents, and its associated bureaucracy) influences costs and frames the startup business model.”

Even if not explicitly indicated in the diagram, we cannot forget that many elements in the ecosystem map play the role of connectors. Universities, for example, provide not only knowledge, but also a favorable environment for deep, long lasting connections. Investors do not serve only to put money in business, but much more to connect growing companies with other players. Mentors, accelerators, families, and events also play a fundamental role on connecting people.

Phase 2 results: insights from the São Paulo ecosystem

The objective of the second phase was to analyze the São Paulo startup ecosystem and use the insights about this ecosystem to validate and refine the conceptual model of phase 1, as well as to propose a model for the ecosystem maturity process.

The overall conclusion was that, although the São Paulo region and its major university, USP, have enormous potential for entrepreneurship and innovation, mainly due to its human capital, the current status of the ecosystem is still weak when compared to the most advanced ecosystems in the world. The region is not capable of generating significant disruptive innovation or producing startups with a global impact. This indicated that, looking from a global, international perspective, this ecosystem is currently in an early stage of maturity.

The major problems in the São Paulo ecosystem identified by our research were (1) lack of connectivity/weak people networking, (2) lack of entrepreneurial culture and prejudice against businesses and applied research within the university, (3) high bureaucracy and lack of flexibility both within universities and in the market legal/tax frameworks, (4) no incentives for professors and students to pursue innovation and entrepreneurship, and (5) lack of high-tech startups (there are a few, but most entrepreneurs in the city do not engage on applying recent scientific advances to their business).

Our analyses identified a few key recommendations for ecosystem leaders, entrepreneurs, university administrators, and policy makers that could significantly change this landscape within a few years:

• 1. In universities, create a vibrant “entrepreneurship lounge” to concentrate robust activities around entrepreneurship and innovation and gather students, alumnae, and professors from all university schools, while promoting local, smaller activities within the various institutes.

• 2. Spread the entrepreneurial culture with short and semester-long courses, events, and incentives for professors, students, and alumnae to engage in innovation activities. Entrepreneurial professors should be valued in their career progression, while students should receive credit for their innovative projects. Facilitate the participation of professors in innovative commercial ventures and the transfer of technology from the university to society.

• 3. Create new policy with incentives for innovative startups and simplify bureaucratic requirements for nascent, tech-based companies.

More details about the results obtained in phases 1 and 2 can be found in the associated technical reports (Kon et al. 2015; Cukier et al. 2016) and Masters thesis (Santos 2015).

After this second ecosystem analysis, and with a more comprehensive understanding of how ecosystems operate and the associated theory, we identified a significant literature and theoretical gap in describing ecosystem dynamics. The main insight gained from the second iteration analyzing a startup ecosystem was the need for a practical method for identifying where each ecosystem is located in its evolutionary process.

Based on the core elements discovered during the case studies of phases 1 and 2, we proposed a first version of a model for describing the maturity process of startup ecosystems. This initial model proposed to organize the evolution process of ecosystems in four levels of maturity: nascent (M1), evolving (M2), mature (M3), and self-sustainable (M4). The criterion for classifying each level was presented in a workshop (Cukier et al. 2015a). Even if this initial model had adherence to the two ecosystems studied until then, we decided to triangulate, performing a third case study in New York City. The objective was to verify whether (1) this ecosystem fitted the model or (2) the model needed refinements.

Phase 3 results: the New York case-study adherence to the maturity model

The first high-growth technology software startups in New York City appeared in the late 1990s in the media industry. Then, the dot-com bubble burst, and New York lacked an established ecosystem, like the one in Silicon Valley, to support its technology startups. The few entrepreneurs who persevered formed the base of the first entrepreneurial generation who later led the ecosystem’s development (Cometto and Piol 2013). New York, the business capital of the world, as well as the center of advertising and the financial, food, and fashion industry, supported by a robust high-tech entrepreneurial policy system and a strong pool of human capital, blossomed into FinTech, FashionTech, FoodTech, AdTech, Marketing Tech, Real Estate Tech, and so on.

We heard from interviewed specialists that building a business is becoming cheaper and cheaper worldwide due to easy access to basic resources (computer infrastructure via cloud services, Software as a Service, Open Source software), and the mobile Internet-connected world. The recent trend towards the Internet of Things also opens up a variety of business opportunities (Russo et al. 2015). Specialists also state that each ecosystem has expertise in a few specific industries. While Boston is a worldwide leader in areas like pharmacy and bio-tech (Herrmann et al. 2015), NYC is strong in media and financial (FinTech) startups.

In the late 1990s, the New York City software startup ecosystem was in its nascent maturity level and had already acquired much of the necessary support infrastructure to evolve quickly: the metropolitan region is home to top research universities like Cornell, Columbia, New York University, and the City University of New York, which all have special programs for entrepreneurs; many (sometimes free) co-working spaces like General Assembly and WeWork (which was valued $17 billion in 2016) started to emerge; the public transportation system is efficient; and big tech companies established offices in the city (for instance, Google’s office in the Chelsea neighborhood).

The Boulder Thesis (Feld 2012) states that a prosperous ecosystem has four characteristics: (1) it is led by entrepreneurs; (2) it is inclusive, such that everyone is welcomed; (3) the people involved are committed long term (at least 20 years) to the ecosystem; and (4) there are many opportunities for gathering, i.e., many events. New York is a perfect observational instance of the Boulder Thesis:

• Even if part of the NYC impetus for the ecosystem came from direct efforts of the Bloomberg Administration (i.e., the Mayor’s Office)—supporting incubators, accelerators, and co-working spaces—entrepreneurs are still the central agents, as everything starts with them. A NYC interviewed entrepreneurship professor remarked: “put two or three great entrepreneurs anywhere and they will create an ecosystem.” The idea that entrepreneurs are the center of the ecosystem challenges the triple helix model (Brannback et al. 2008), built on the idea that the regional innovation hubs emerge only from the collaboration among universities, industries, and governments.

• Events are visible and easily located to bring people together and create a community. The largest tech meet-up in the world, the New York Tech meet-up, has fifty thousand members. In 2014, the City officially launched Digital.NYC, an on-line hub for the startup ecosystem, bringing together almost every startup, investor, event, class, job opening, workspace, accelerator, news story, blog, video, and startup resource. Small events happen every day, medium events happen every week, and large events every month.

• Inclusiveness: everyone is welcome in the cultural mix of NYC. When you look at startups, you find founders from dozens of different nationalities. Additionally, there are twice as many startups founded by women than in Silicon Valley (Cometto and Piol 2013). The NYC Tech meet-up and other events involve people of all ages, with everyone from the elderly to children watching startup pitches.

• Long-term commitment: the first generation of entrepreneurs that led their startups to successful exits and helped to found the New York Angels, and many other investment groups, are a good example of persistent commitment, as is the Cornell Tech program, which was planned between 2009 and 2011. The Cornell Tech program’s intent was to create high-tech and applied science entrepreneurship courses and build a new campus on Roosevelt Island. The construction project began in 2014, it opened in late 2017 and will be complete in 2037. It received donations from many Cornell alumni, including a very large one from Charles “Chuck” Feeney of US$350 million (Pérez-Peña 2011) and one of US$133 million from Joan and Irwin Jacobs to fund the Technion-Cornell Institute.

The combination of entrepreneurs’ long-term commitments with the many inclusive startup events leads to a highly connected ecosystem. Theory shows that successful innovation ecosystems depend on a high level of inter-connectivity among its players (Breznitz and Taylor 2014; OECD 1997; Iansiti and Levien 2004). NYC has several highly connected entrepreneurship networks, and successful business such as DoubleClick, which was bought by Google, have created a network effect. Howard Morgan, one of the founders of New York Angels, explains why: “exits like that (US$ 1.1 billion) are essential to grow the high-tech ecosystem, because managers and engineers who have made some money (...) can then take some risks in other startups” (Cometto and Piol 2013).

A recent industry report for startup ecosystem rankings shows the New York City ecosystem evolving from the fifth place in 2012 (Herrmann et al. 2012) to the second place in 2015 and 2017 (Herrmann et al. 2015; Startup Genome 2017). Another report put New York in the first place in 2015 (Cain 2015). If these reports existed before, and considering the criterion they used, it is likely that New York would not be among the top ranked ecosystems before 2009.

Many theoretical models about entrepreneurship regions point to culture as a highly important dimension when analyzing these ecosystems (Hofstede et al. 2010; Macke et al. 2014; Lyons et al. 2012). We observed a cultural shift in New York City after the 2009 crisis. Many interviewees commented that, before the crisis, highly qualified engineers in the financial market were comfortable with the salaries their employers paid. When the market crashed, many tech talents lost their jobs and realized that they were not as safe as they believed. The opportunity cost of starting a new company seemed smaller, and taking the risk was no longer as big of an issue. Because of the traditional financial market crash, many investors began to look for new investment opportunities. Moreover, the financial district office spaces were completely empty and rental prices decreased. To promote the recovery of real estate, financial district owners offered free co-working space for new startups, with the hope that their growth in the future could bring more real estate business to the district.

The financial crisis in 2009 also impacted on people’s decision to invest time in high-level education. In fact, “New York has more college students than there are people in Boston”, affirmed one of the interviewed entrepreneurs. Another entrepreneur emphasized his decision to pursue a PhD because of the crisis: “when the market crashed, I didn’t know what I was going to do, then an opportunity arose to do a PhD and then I thought that was great because there was nothing else to do”.

By analyzing raw data from Crunchbase, one of the largest and most complete startup databases in the world, we created two graphs that show the evolution of the New York ecosystem. Looking at Fig. 4, we observe an explosion of new startups being created in New York since 2009. The same degree of growth happened to the number of companies that got their first investment. Figure 5 shows that, even if the number of initial public offerings (IPOs) remained static within the ecosystem, the number of acquisitions grew at the same pace as the creation of new companies or investment deals.

Fig. 4

Companies founded in New York and number of first investment deals per year. The number of founded companies in 2016 is lower because companies normally appear in Crunchbase only after the first investment round, which seldom occurs in the first 2 years of existence. We show numbers up to 2016 because 2017 numbers are still scarce. Source: Our graph from raw Crunchbase data

Fig. 5

New York startup acquisitions and IPOs. We show numbers up to 2016 because 2017 numbers are still scarce. Source: Our graph from raw Crunchbase data

In New York City, the very first technology startups appeared in the mid-1980s, but it was not until the mid-1990s that New York’s first high-growth startups began to emerge in the media industry. The ecosystem experienced a modest growth during the 2000s, passing from a nascent stage to an evolving stage during these years. This first evolution was observed in the following metrics of the maturity model (see the “The startup ecosystem maturity model—final version” section): a growing number of events, the first large startup exits, and the first specific university programs. In the beginning of the 2010s, it reached the mature stage (increased number of mergers and acquisitions (M&A) and IPOs, a second generation of entrepreneurs, and growing angel investment groups) on its way to reaching a self-sustainable stage in the last couple of years.

We observe this evolution not only in the number of startups and investment deals, but also in other factors, such as event frequency, support from big tech companies (like the first Cornell Tech course in Google’s office), and co-working spaces. After 2010, the ecosystem started the path toward a virtuous cycle when the older generation of successful entrepreneurs became angel investors or serial entrepreneurs. In addition, after 2012, the number of startup acquisitions per year exploded (almost one every 3 days), indicating a prosperous environment for investors and entrepreneurs. It is worth noting that, since 2011, the number of new companies did not grow as much as the number of investment deals or acquisitions. This shows a tendency of abundance in access to funding in the last years, or a saturation of talent availability, suggesting that the ecosystem has space for more startups and needs more investment to attract and retain talent.

In the next section, we present the final version of the maturity model and, after that, we show the answers to our research questions.

The startup ecosystem maturity model—final version

The conceptual framework developed in phase 1 and refined in phase 2 contained the core elements of every software startup ecosystem. The phase 2 refinement led us to a theoretical saturation of the framework, after which no new core elements were discovered. These core elements are not isolated characteristics; rather, they relate to each other in different ways. For each core element, it is possible to analyze its level of development, as well as the quality of the relationships between them, to measure the degree of maturity in each aspect. Examples of maturity metrics in the core elements include the existence of funding bodies, the development level of the funding structure, the presence of technical talent provided by high-quality educational institutions, and access to educational resources.

To devise the maturity model, we transformed the core elements of the conceptual framework created using the Ground Theory (see Fig. 3) into a list of metrics. We now present a description of each metric, an indication of how we measure it, and its relationship with the core elements in the conceptual framework.

• Exit strategies—Entrepreneurs and investments are considered successful when one of the following happens: (a) profitable growth to the global market, (b) acquisition by a big company, (c) merge with another company, or (d) IPO. Especially for investors, the existence of exit options in the local ecosystem is an attractive factor. While mature ecosystems present all four strategies, there is a lack of exit options in new ecosystems. Zero options is considered weak, one option is medium, and two or more options is a sign of maturity. Related framework elements: startup, funding bodies, and established company.

• Global market—Percentage of startups that targeted the global market. A startup is considered to target the global market if it acts in markets outside its country, with existing customers or at least an official representation office. Related framework element: market.

• Entrepreneurship in universities—Percentage of alumni that founded a startup within 5 years of graduation. Related framework elements: universities and research centers, education.

• Mentoring quality—The percentage of mentors that fit one of these criteria: (1) had a successful startup in the past and (2) founded and worked for more than 10 years in one or more startups. Related framework elements: entrepreneur.

• Bureaucracy—Based on the inefficient government bureaucracy index of the global competitiveness report (Schwab 2013). It represents the percentage of respondents that considered bureaucracy as a problematic factor for doing business. Related framework elements: legal frame.

• Tax burden—Based on the country’s total tax rate ranking of the global competitiveness report (Schwab 2013). Related framework elements: legal frame, market.

• Accelerators quality—Percentage of startups in accelerators that reach the stage of receiving a next level investment, or reach the global market in a sustainable profitable stage. Related framework element: incubator/accelerator.

• Access to funding in US$—Total amount of investment in startups in US$ according to a trusted database. Related framework element: funding bodies.

• Human capital quality—Based on the ecosystem position in the talent index of the global startup ecosystem report (Herrmann et al. 2015). Related framework elements: entrepreneur, education.

• Culture values for entrepreneurship—Cultural support index in the global entrepreneurship and development index (Acs et al. 2015). Related framework elements: culture, society, family.

• Technology transfer processes—Based on innovation and sophistication factors of The Global Competitiveness Report 2013-2014, p. 22 (Schwab 2013). Related framework elements: university/research center, legal frame.

• Methodologies knowledge—Percentage of startups that have knowledge or are trained on systematic methodologies¹¹. Related framework elements: methodologies.

• Specialized media players—Local media specializing in the startup industry plays an important role in spreading the word about what is happening in the ecosystem. The existence of more than five players is a sign of movement and engagement within the ecosystem. The specialized media must be recognized by the local community as a reference to be considered in this list. Related framework elements: media.

• Startup events—How frequently local events focused on themes like high-tech entrepreneurship or startups occur. Related framework element: society/events.

• Ecosystem data and research—The existence of a database with data about the ecosystem is an indication of maturity. It is more difficult to improve what one cannot measure; thus, ecosystems that do not have research institutions nor metrics cannot recognize the next steps to take. Related framework elements: research center, government.

• Ecosystem generations—the number of generations of prior entrepreneurs that are re-investing their earnings in the ecosystem. “0” means no prior entrepreneurs are investing in the ecosystem, “1” means a first generation of prior entrepreneurs re-investing their earnings in the ecosystem, “2” means that entrepreneurs that received investment from generation 1 are investing their earnings in new startups, and so on. Related framework elements: entrepreneur, society.

• Number of startups—Quantity of startups founded by year, according to a trusted database. Related framework elements: startup, market, entrepreneur.

• Access to funding in number of deals/year—Deal count, independently from value or startup stage. Related framework element: funding bodies.

• Angel funding in number of deals/year—Deal count only by Angel investors. Mature ecosystems tend to have more angel investment support, since angels are usually successful entrepreneurs giving back their earnings to the community. Related framework element: funding bodies/angel.

• Incubators/tech parks—The number of incubators and tech parks active in the ecosystem. Related framework element: incubator/accelerator.

• High-tech company presence—How many high tech companies have tech teams located in the ecosystem region. Related framework elements: established companies.

• Established companies influence —How many big companies have activities that nurture the ecosystem? Activities include event organization, local community ambassadors and mentors, acceleration programs, or local investments in startups. Related framework elements: events, established companies, startup, accelerator, and entrepreneur.

Thus, we proposed, a scale to evaluate each factor’s state. The scale contains three levels of development: L1, L2, and L3. This scale was generated after a series of iterations with specialists and confirmation of what they considered appropriate measurement of L1, L2, and L3 in each aspect. We then proposed a metric to classify ecosystems for each core element maturity.

Some factors in the ecosystem comparison table are crucial to consider when an ecosystem has reached a certain level of maturity. Not achieving a specific grade in any of these factors stalls the ecosystem at a lower level of maturity. Thus, we divided the factors into two categories: essential and complementary. The complementary factors are important to “upgrade” the ecosystem to the next level.

Our proposed maturity model comprises four levels, as described below with a short description of each level:

• Nascent (M1): usually a nascent ecosystem is already recognized as a startup hub, with already some existing startups, a few investment deals, and perhaps government initiatives to stimulate or accelerate the ecosystem development, but no great output in terms of job generation or worldwide penetration.

• Evolving (M2): ecosystems with a few successful companies, some regional impact, job generation, and a small local economic impact. To be at this level, the ecosystem must have all essential factors classified at least at L2, and 30% of complementary factors also on L2.

• Mature (M3): ecosystems with hundreds of startups, where there is a considerable amount of investing deals, existing successful startups with worldwide impact, and a first generation of successful entrepreneurs who started to help the ecosystem to grow and be self-sustainable. To be in this level, the ecosystem must have all essential factors classified at least at L2, 50% of complementary factors also on L2, and at least 30% of all factors on L3.

• Self-sustainable (M4): ecosystems with thousands of startups and financing deals, at least a second generation of entrepreneur mentors, especially angel investors, a strong network of successful entrepreneurs engaged with the long-term maintenance of the ecosystem, an inclusive environment with many startup events, and high-quality technical talent.

  To be at this level, the ecosystem must have all essential factors classified as L3, and 60% of complementary factors also on L3.

Important to notice that, even if the maturity levels are numbered in a linear scale, it is easy to see that the dimensions grow in an exponential manner. Thereby, for example, assuming that the multiplication factor is 10, an ecosystem in level 3 is 100 times more evolved than another ecosystem in level 1.

By analyzing the ecosystem maturity model (Cukier et al. 2015b; 2015a) proposed on phase 2 with New York observations, as well as feedback from specialists at the PROFES’2015 Software Startups Workshop, we refined some of its proposed factors. For example, we replaced the absolute number of startups in an ecosystem with the relative number of startups per million inhabitants. In fact, all factors that considered absolute numbers were revised to use metrics that are relative to the ecosystem size. Otherwise, ecosystems outside very large cities would never reach their maturity if the criterion of having high absolute values was kept. We also received assistance and valuable feedback from one of the major authors of the Startup Ecosystem life cycle model, a similar approach to design and map ecosystem evolution (Gauthier et al. 2015).

Two new essential factors were added to the maturity model:

• 1. Access to angel funding. “Startup communities feel like they are not complete until they have at least one angel investor group” (Cometto and Piol 2013). Many interviewees mentioned the great importance of role models. For technology startups, it is important that these role models include not only successful business people, but also developers and tech leaders. One of the entrepreneurs interviewed said, “I think it is essential to give a lot of equity to engineers. When a company becomes successful and these guys become rich, they are the ones who will start new innovative companies.” Successful tech founders will not only inspire new entrepreneurs, but also become angel investors for the next generations.

• 2. Events: it is nearly unanimous among the New York City interviewees that social networking spaces and events are important to the ecosystem’s maturity.

Another modification we made to the model was to remove access to funding from the essential factors list. Even if access to funding is very important, it is more a side effect than a cause of success for ecosystems. One investor said: “When you have amazing technology companies being created anywhere, investors will follow.” We present the final version of the model proposal in “The startup ecosystem maturity model—final version” section. We removed the military metric, which was considered very specific to only a few ecosystems, and added new metrics to the model: startup events and angel funding in number of deals/year.

After generating the classification table for each factor, we filled in the table with data about the three ecosystems we analyzed (Tel Aviv, São Paulo, and New York), using the help of two specialists from each ecosystem. If we consider the New York ecosystem evolution from 2000 to 2015, the ecosystem passed through all four stages proposed in the maturity model. Moreover, some initiatives began by stakeholders when the ecosystem was considered nascent or evolving (such as the creation of the New York Tech meet-up), and other events that happened later when the ecosystem was already considered mature (such as the Cornell Tech project (Pérez-Peña 2011)), all came at the right moment for the ecosystem, helping it to evolve quickly and robustly. Based on our research in Israel (Kon et al. 2014), we consider that the Tel Aviv ecosystem is in the same self-sustainable stage as New York and observe that it took similar evolutionary steps. Our research in São Paulo concludes that this ecosystem already passed the nascent stage, but is still at the evolving stage. The characteristics and dynamics of all these three ecosystems fit the proposed maturity model. We chose Tel Aviv to analyze an evolved ecosystem outside the US market and avoid bias on US culture-specific characteristics. The choice of São Paulo, besides being the ecosystem two of the authors were immersed in, derived from the importance of investigating ecosystems in a more immature stage (from a global, international perspective), in a developing country, and understanding the specific needs in this context.

The startup ecosystem maturity model is depicted in Table 2.

Table 2 Ecosystem maturity model factor classification: final version

For ease of understanding, and to facilitate dissemination, we created a summarized version of the model depicted in Table 3. The level classification using the summarized version requires that the analyzed ecosystem must have at least seven (from eight) factors classified on that level.

Table 3 Ecosystem maturity model: summarized version

It is worth noting that, depending on the ecosystem maturity level, the metrics have a different level of importance. Some metrics are more important to measure and develop during the first years of the ecosystem development, while others are more relevant when the ecosystem has already achieve an advanced level of maturity. Table 4 summarizes the metrics-importance classification. This table indicates where local agents should focus their efforts in the ecosystem development process.

Table 4 Ecosystem maturity model: metrics importance

Answers to research questions

The existence of a maturity model helped us to find the answers for our four research questions.

A. What are the minimum requirements for a startup ecosystem to exist in its nascent stage?

Fred Wilson, co-founder and managing partner of Union Square ventures, claimed that “the story of NYC is a story of entrepreneurship (...), entrepreneurs re-investing their wealth back into the next generation of entrepreneurs. (...)” (Cometto and Piol 2013)—it follows that one of the first requirements for an ecosystem to exist is to have great entrepreneurs. It seems obvious that any entrepreneurial ecosystem needs entrepreneurs, but it is not so obvious that the entrepreneurs are the seed of everything. This means that talented entrepreneurs are necessary even at the first nascent stage of an ecosystem.

The existence of high-quality research universities in the region is an important attractor for these talents, especially when there are programs for tech-entrepreneurship. The presence of big tech companies can also be considered a talent attractor, but not necessarily the talents that will become entrepreneurs.

By analyzing the three ecosystems in our case study, Tel Aviv, São Paulo, and New York, it is clear that all of them surpassed the nascent stage.

B. What are the requirements for a startup ecosystem to exist as a mature self-sustainable ecosystem?

Startup ecosystems reach a mature self-sustainable level when there are at least three generations of successful entrepreneurs that start re-investing their wealth in the ecosystem by becoming angel investors and offering their mentorship. This is only possible when there are many opportunities for M&A and IPOs in the market, and, moreover, when the entrepreneurial culture is widely accepted and understood, supported by high-quality educational institutions, and startup events happen almost every day. When the ecosystem reaches the self-sustainable maturity level, the media also plays the role of maintaining the momentum and awareness of the public.

In our case study, both Tel Aviv and New York are considered to have reached the self-sustainable M4 maturity level. On the other hand, São Paulo has not reached this stage yet, since we do not observe there the required characteristics, such as an evolved IPO market or three generations of successful entrepreneurs. The classification in Table 5 also emphasizes the maturity level for the ecosystems analyzed in our study.

Table 5 Startup ecosystem comparison table

C. What are the stages that ecosystems pass through? Can they regress or die?

Our interviews, observations, and feedback from experts led to the definition of four stages that ecosystems pass through: nascent (M1), evolving (M2), mature (M3), and self-sustainable (M4). The transition between stages is smooth and may take years. The classification is sometimes fuzzy, especially during the transition between phases.

It is possible that startup ecosystems can regress, but it is rare. An angel investor and serial entrepreneur explains: “the ecosystem evolution is a one-way street, because created conditions are self-reinforced”. “Very drastic situations like wars or natural disasters can eventually lead the ecosystems extinction” – said another entrepreneur: “These are very rare situations, thus the natural path is evolution” (Breznitz and Taylor 2014). We would add persistent economic crisis in the country in which the ecosystem emerges as an additional threat to its evolution.

In our analysis, Tel Aviv and New York passed through all four stages of evolution in the last 50 years, from nascent to self-sustainable, while São Paulo, a younger startup ecosystem, which had it first significant tech entrepreneurs’ generation around the 2000s, is still in the evolving stage.

D. Can people proactively interfere in the evolution of ecosystems? Is it possible to exist other ecosystems as developed as mainstream ecosystems such as Silicon Valley, generating tens of high growth global startups? How many of these could exist in the world?

Fred Wilson adds, “what has happened in NYC can happen anywhere that has the entrepreneurial spirit and the freedom to innovate” (Cometto and Piol 2013)—and many interviewees agreed that it is possible for self-sustainable ecosystems to exist if the local culture values the entrepreneurial behavior, confirming Brad Feld’s claim that “You can create a vibrant long-term startup community anywhere in the world” (Feld 2012). Two interviewees compared New York with Boston, claiming that Boston’s ecosystem (the home of MIT) did not take off as fast as New Yorks, because Boston’s culture is much more conservative, while New Yorkers are more open to risk. Thus, on the one hand, people can interfere to accelerate; but, on the other hand, culture, which is something very difficult to change in the short term, plays a very significant role.

Nevertheless, we learned from our research in São Paulo that culture can change, though it may take time. There, the first generation of tech entrepreneurs started timidly in 2000. At that time, young people were supposed to finish their university degrees and find a job in a large company. After 15 years, the scenario changed to a culture in which being an entrepreneur is a lifestyle. São Paulo is a city with many characteristics similar to New York: a large metropolis, with millions of people (mostly first-, second-, and third-generation immigrants); a financial, advertising, and business center; and a culture of hard work, where time is money. São Paulo has all the potential to evolve from the current evolving (M2) level to mature (M3) or even self-sustainable (M4), but for that to happen, it must overcome important obstacles, like developing more policies for tech-talent attraction, reducing the tax burden, improving the law framework for company creation and closing, investing in mobility infrastructure to facilitate the access to high-quality universities, and developing the investment market.

Threats to validity

Maxwell identified potential limitations of qualitative research (Maxwell 2012). Descriptive validity happens when the research is able to accurately collect the data. In our case, most of the interviews were tape recorded, so we could later listen to the audio. Moreover, we had some informal conversations with ecosystem agents, so not all of our interactions were under rigorous scientific control.

Interpretation validity occurs when the investigator has no influence over the interviewee answer, avoiding to guide the person to a desired response. Our interview protocol, which was rigorously followed, made very clear that the interviewer should speak as little as possible, making short questions and letting the interviewee express him/herself as freely as possible. Nevertheless, in our case, questions already contained concepts and vocabulary such as “startup ecosystem,” “ecosystem agents,” and “maturity model,” influencing the interviewee to believe that these paradigms were established. The interviewees’ responses can be biased by the few concepts about ecosystems presented to them during the interview.

Researcher bias happens when the investigator focus only on his/her own previous background to the research conclusions, avoiding to include in the research offenders of the preexisting beliefs. By using a Grounded Theory approach, we avoided part of this bias. However, while the first interviews are less biased by our previous knowledge, one limitation of this research is that the last interviewees are biased by the preconceptions we acquired on previous interviews. Moreover, people are somewhat biased by what they see on media about what is considered a successful path today for technology startups: founding a company, raising seed capital from angels, raising series investment from VCs, becoming a unicorn (US$1 billion company), going public, or being acquired. The media emphasizes only the successful cliché cases, and entrepreneurs are overwhelmed by these idealized models. Usually, the media ignores medium and small local companies that may also have an important impact on the startup ecosystem and on regional development. Their influence and role were not considered in our study and need to be further investigated.

Theory validity happens when the researcher does not force the data to match a previously developed theory. The Grounded Theory techniques we used help to avoid this threat during the elaboration of the startup ecosystem conceptual model. On the other hand, after phase 1, we dove into the literature to support the next phases, so even if the conceptual framework for software startup ecosystems is unbiased by previous theory, the maturity model is influenced by the existing body of literature.

Reactivity occurs when the collected data and observations are taking place just because the investigator is there. The first part of our interview protocol was built to avoid this kind of phenomena, specially when we did “breaking the ice” questions, trying to make the interviewees comfortable. At a first moment, interviewees may get intimidated by the recording equipment, but after some minutes they forget that they are being recorded and begin to act more naturally. A small part (approximately 20%) of the interviewees already knew the researchers before the interviews and had a previous relationship with them. These interviewees may have their answers influenced, e.g., avoiding to offend the interviewer’s opinion. Still, we believe we performed a sufficient number of unbiased interviews, specially those with previously unknown interviewees.

A last validity threat to our research is that the proper choice of studied cases is an intrinsic limitation of multiple case studies (Stake 2013; Yin 2013). Even if we composed our choices with cases that are considered complementary, we understand that we limited our choices to western startup ecosystems. We believe that further efforts are required in investigating Asian, Eastern European, African, and Australian ecosystems to understand if these cultures also fit into the proposed maturity model. Also, we focused our cases on large urban centers. It would be valuable to verify whether the model would work well for startup ecosystems in smaller cities such as Boulder (CO, USA), São Carlos (Brazil), Trondheim (Norway), or Bolzano (Italy).

Conclusions

With the results obtained from this research, we attained our general objective to advance the understanding of how software startups and their ecosystems function. We presented a conceptual framework that depicts software startup ecosystems, their agents, and the relationships among them, accomplishing the specific objective 1. We iteratively evolved this framework until reaching a final version that fitted well in all three analyzed ecosystems (specific objective 2). We created a maturity model to map ecosystem evolution, achieving the specific objective 4. Stakeholders from any existing ecosystem can use the proposed model to evaluate its maturity and also compare to others (specific objective 4).

The New York case is a strong example of how startup ecosystems can evolve over time. In 2010, this ecosystem had a very modest impact regarding startup creation and innovation generation compared to other ecosystems, such as Silicon Valley, Boston, or Tel Aviv. Less than 5 years later, the New York City ecosystem is considered a benchmark: the best place for startups according to the CITIE 2015 Report (Cain 2015), and the second best in the Global Startup Ecosystem Ranking (Herrmann et al. 2015; Startup Genome 2017).

By studying Tel Aviv, São Paulo, and New York, three completely different realities, we observed that, along time, these ecosystems passed (or are passing) through the same stages of evolution. High-tech entrepreneurial ecosystems in different countries are composed of the same agents (entrepreneurs, society, government, universities, funding bodies, etc.), and the interdependencies and relationships across these agents occur in a similar manner.

In future work, we would like to collaborate with other researchers in using the maturity model to analyze new regions and derive concrete actions that should be taken to improve those ecosystems. Some questions remain: Is there a limit to how many self-sustainable ecosystems can exist? To what extent does local culture influence the appearance of these ecosystems, since it is a limiting factor for all other aspects of the model? Since theory emphasizes the importance of the ecosystem connectivity, new research should focus on ways to measure an ecosystem’s connectivity based, for example, on online social network data.

One could also extend our research to other regions outside of large urban centers. It is a challenge to develop fruitful startup ecosystems in smaller towns such as São Carlos, São José dos Campos, or Campina Grande in Brazil, or cities like Trondheim in Norway or Bolzano in Italy, or maybe even cities in Africa and the Middle East. On the long-term, small and medium cities tend to lose talent and resources to the big centers. We consider that there is a vast field of research to be explored about startup ecosystems in small and medium cities.

We created the maturity model in the context of software startups and their ecosystem. This does not mean that the model is useful only for startups based on software. Many conclusions could also be applied for startups around hardware, biotech, and other technologies. Today, it is rare to find high-tech startups that do not have any software in its core. Even hardware companies often need software to scale their business. Nevertheless, further research over other sectors could enhance the model.

Even though we achieved theoretical saturation in the conceptual framework, we believe that it is a moving target. Things are always changing over time, so the conceptual framework certainly needs to be revisited from time to time. After being applied in five or ten other ecosystems, one should consider new adaptations to the maturity model.

Ecosystem agents should work together in collaboration, with a shared understanding of the complex structures in which they are embedded. We hope that the research described in this paper brings valuable insights for entrepreneurs, governments, investors, established companies, and other stakeholders in any innovation ecosystem. We believe in technological innovation as a road for improving human life and hope this work will contribute to the journey within this long and fascinating road.