Skip to main content

Assisting regional policy by rapidly comparing enterprise innovation between regions



Regional development must heavily rely on innovation in order to be competitive and improve the standard of living of its citizens. However, regional policies regarding innovation is often limited by decelerated decision making process which is often based on outdated qualitative information and data which are difficult to statistically analyze. In order to continuously evaluate the progress of regions and assess different innovation indicators in shorter periods, a quantitative measurement of innovation should be followed.


To this end, we collected data from firms belonging to two geographically different regions in order to develop, present, and evaluate a research approach and present the statistical methods which can be applied in order to benchmark innovation strengths and weaknesses between different regions and apply consequent policy actions.


Both our research approach and application of the methodology successfully examined the usage of an applicable research instrument and nonparametric statistical analysis to enhance data collection used in regional policy development related to innovation. We present the application of these tools and the results of the comparison as a test case.

Research limitation/implications

Our study, being an initial effort to develop a simple rapid tool and provide the methodological background to measure and compare regional enterprise innovation, is limited considering the number of innovation characteristics examined and the applicability of our research tool and methodologies on a single pair of regions.

Practical implications

This paper focuses on examining the ways the data are collected through the measurement of common innovation characteristics that can be easily and validly analyzed in order to aid regional innovation policy makers.


A significant consideration for both researchers and policy makers is the relatively low frequency of data collection based on surveys. We propose a quantitative measurement of innovation, focusing on the benchmarking of innovativeness of firms, which can be applied easily in practice, in order to benchmark the innovation strengths and weaknesses between different regions and apply consequent policy actions.


The innovation of regions is becoming a crucial issue in regional development. The current state of the world economy is influenced by globalization process, urging individual states and regions to cope with its impacts and create conditions for the continual restructure of their own economies. Building the capacity to restore unemployment rate to acceptable levels, reverse the economic and societal crisis, and establish a higher future standard of living depends on the regions’ ability to drive innovation in products, services, businesses, and social processes. The main drawback for regions in the European Union (EU) member states, as recognized by EU Communication (COM-2010-553), is the limited innovation capacity of businesses, associated with low research and development levels, and scarce links with universities and research centers. The ability of a region to excel heavily depends on the innovation capacity of enterprises that reside in the region, and thus, their role is critical.

In order to increase the regional level of innovation, the innovative capacity of its enterprises has to be measured and evaluated so as to identify limiting factors and create favorable conditions for innovation, education, and research and increase research and development activities, knowledge-intensive investment, and provision of higher value added activities.

Yet, measurement of innovation has been recognized to be not an easy task, a weakness most possibly caused by both methodological shortcomings and the unpredictability of innovation outcomes (Tidd 2001). However, in the current economic context, measurement appears as a critical discipline.

Measurement of innovativeness in terms of various units has become the focus of various methodologies such as the European Innovation Scoreboard (2008) (EIS), the Regional Innovation Scoreboard (RIS), and the Community Innovation Surveys (CIS), being the main statistical instruments of EU for measuring innovation activities at the national, regional, and firm level, respectively. Additionally, simple or composite indicators based on information from the evaluated units have been the focus of several studies, as described in the work of Archibugi et al. (2009) and Nardo et al. (2008).

Despite the increasing amount of research on measuring innovation, several issues regarding the collection, analysis, and exploitation of innovation data are still unresolved. A significant consideration for both researchers and policy makers is the relatively low frequency of data collection based on surveys (like EIS, RIS, CIS, etc.) that are scheduled periodically, in yearly or more extensive periods. More recurrent innovation data would be very useful in order to explore the progress of the evaluated regions and assess the different innovation indicators in shorter periods. This is also deteriorated by difficulty in analyzing statistically qualitative (and not quantitative) information and data.

In view of these requirements, we propose a quantitative measurement of innovation, focusing on the benchmarking of innovativeness of firms belonging to two geographic regions, with diverse innovation characteristics. For this purpose, we collected data from firms belonging to the commerce sector in two geographically different regions of a single country in order to develop, present, and evaluate a research approach, which can be applied easily in practice, in order to benchmark innovation strengths and weaknesses between different regions and apply consequent policy actions. This paper does not solely rely on the development of such a tool, a task undertaken by researchers, but mainly on the ways the data collected through the measurement of common innovation characteristics can be easily and validly analyzed in order to produce results that can become of great assistance to regional innovation policy makers.

In the next section, we analyze previous research on innovation measurement, while in ‘Research methodology’, we present our research methodology and mainly the statistical tests that provide a valid and easily applicable tool in comparing the innovation capacity of the enterprises that reside in the region. In the section ‘Applicability of the methodology to specific regions’, we present the results of applying these tools in two distinct geographical regions. Next, we present conclusions and implications for researchers and regional policy makers, and in the last section, limitation and future pathways for this research are presented.

Previous research

In this context, measurement appears as a critical discipline within the literature on innovation management. Thus, a wide range of measures has been proposed and tested empirically to assess the degree of a company's innovative ability (Barclay 1992; Kim and Oh 2002), while the relationship between innovative ability and business performance (i.e., between inputs and outputs) has been widely examined at both the industry level (Cooper and Kleinschmidt 1991; Guan et al. 2009; Huff 1990; Sorescu et al. 2003;) and the firm level (Adams et al. 2006; Sánchez et al. 2011; Tidd 2001).

Focusing on the firm level, Tidd (2001) attempts to aggregate and classify the typical parameters found in literature as measures of business performance. Hence, he identifies two approaches to measuring innovation: the one utilizes indicators available in the public domain, such as research and development (R&D) expenditure, number of patents, and new product announcements, while the other uses survey instruments to capture a broader range of indicators such as the proportion of technical, design, or research personnel, proportion of sales or profits accounted for by products launched in the past years, etc. Moreover, Tidd (2001) provides a list of the main measures, supported by a weaknesses-strengths analysis. However, the key outcome stemming from this analysis is that there is no single optimal measure of innovation. In a more general view, there exist two broad classes of performance measures: the first is concerned with accounting and financial performance (using indicators such as profitability, profit per employees, return on investment, return on assets, share value, etc.), and the second with market performance, usually examining market share or sales growth.

Debating the stream of research presented above, other researchers characterize this treatment as fragmented (Adams et al. 2006). The related criticism provoked focuses mainly on the fact that measuring innovative ability as the monetary input to a process (e.g., R&D spending) or as the immediate output or results (e.g., number of new products developed, speed to market, new products as percentage of sales, number of patents generated, etc.) provides a rather narrow and unbalanced view of the innovation management phenomenon. This failure is the result of having the technological aspects of innovation as focal point while neglecting the actual processes that turn spending into results (Cordero 1990; Sánchez et al. 2011). These processes deriving from internal capabilities and good innovation management practices address also other types of innovation, apart from the technological one. These types are considered to stimulate the mid- and long-term effects of innovation and capture both short- and long-term aspects of value creation in the firm (Phelps 2004).

The recent work of Sánchez et al. (2011), supported also by previous publications (Adams et al. 2006; Cordero 1990; Hult et al. 2004), concludes that the multidimensional character of the ‘in between’ processes puts a major obstacle to the effort towards the development of a comprehensive and conclusive framework that would facilitate their optimal management. Indeed, a generalized measurement framework would be an extremely useful tool in the hands of managers and would allow them to monitor and evaluate their innovation processes, diagnose weaknesses and limitations, and prescribe remedies (Adams et al. 2006; Cebon and Newton 1999).

Such a framework proposing a sufficient bundle of measures that capture (1) how companies adopt systematic innovation practices at the company level, (2) the complex processes that influence the organization's innovation capability, and (3) the relationship between innovation practices and company success in the mid- and long term has not been shaped yet, still remaining a challenge. Nonetheless, several authors, in their attempt to extend measurement theory and practice beyond the focus on output performance, have suggested concrete assessment areas to set the basis for a general measurement framework.

Such efforts are the ones made by Adams et al. (2006) and Nada et al. (2010). The ‘Innovation Management Measurement Framework’, a seven-dimensional conceptualized framework (Adams et al. 2006) motivated by the general observation that ‘innovation measurement does not appear to take place routinely within management practice and that, where it does, it tends to focus on output measures’ focuses on areas such as inputs, knowledge management, innovation strategy, organization and culture, portfolio management, project management, and commercialization. οn the other hand, Nada et al. (2010) propose an alternative innovation assessment approach which integrates indicators from four strategic perspectives: finance, customer, processes, and learning.

Both aforementioned studies suggest a set of areas that need to be measured in order for managers to gain a deep insight into an organization's holistic ability to manage innovation, taking into consideration its application to their own particular context. As one may observe, they exhibit a number of similarities and could be considered as complementary, in the absence of a more integrated framework.

Several studies have tried to measure and evaluate entrepreneurial innovation by using questionnaire-administered surveys, as presented in the work of Nikolaidis and Bakouros (2009), including the frequently cited papers of Chu and Khosla (2009), Liao et al. (2008), and Tan et al. (2008). Based on this research, more recent studies such as that of Eom and Lee (2010) conducted an empirical analysis to identify the determinants of industry - university and industry - public research institute cooperation, and its impact on firm performance. To do so, they utilized data from the 2002 Korean Innovation Survey, which includes firm-level data on technological innovation in the manufacturing sector. De Jong and Freel (2010) also added to the literature by exploring the connection between firms' absorptive capacity and the geographical distance to their collaboration partners. They hypothesized that investment in absorptive capacity may help compensate for the lack of geographical proximity in innovation-related collaboration. Using survey data on 316 Dutch high-tech small firms, they confirmed their hypothesis. Finally, the aim of Raymond and St-Pierre (2010) is to clarify the impact of R&D on innovation in SMEs by conceptually and operationally distinguishing between product and process R&D, and between product and process innovation, and by integrating these concepts in a research model that specifies their interrelationships. This model has been empirically tested with a survey conducted in 205 Canadian manufacturing companies.

However, limited research has been enacted on integrated statistical analysis for comparing regional innovativeness and entrepreneurship between regions with distinctive geographical and economic characteristics. More often, research is based on analyzing qualitative and policy making characteristics to compare innovative characteristics of different regions, like the work of Guillaume and Doloreux (2011) that examined the innovation in two ‘satellite’ regions in France and Québec. Mainwaring et al. (2007) compared patent holdings of firms in the Celtic economies (Scotland, Wales, and the Republic of Ireland) and a representative English region, South West England, using descriptive statistics and regression. The purpose of Herrera and Nieto (2008) was to analyze whether regional differences exist in relation to factors that bear an influence on the firms' chances of obtaining national innovation subsidies, as well as on the effect of such a policy. They undertook a comparative analysis between the autonomous regions of Madrid, Catalonia, and the Basque Country, using the nonparametric approach of propensity score matching. Finally, Geisler (2000) and Grupp (1998) conducted a wide-ranging review of science and technology indicators and presented valid measurements of various stages of the innovation process.

To this end, the need for an easily applicable but valid innovation measurement process that can be applied to different regions with different characteristics in order to monitor the evolution of entrepreneurial innovativeness is important. After all, using more sophisticated techniques in decision making for urban and regional areas, such as scenario planning, is gaining ground, as seen in the research of Gunnarsson-Östling and Höjer (2011). This tool can become crucial for both researchers and policy makers in order to facilitate regions in developing conditions for the continual advancement of their economies.

Results and discussion

Applicability of the methodology to specific regions

Since the assessment of innovation is a complicated task, especially in the phase of gathering sufficient data by a significant number of firms, most regional policy makers must mainly rely on secondary and archival data. However, to assess true entrepreneurial regional innovation, it is important to rely on primary and up-to-date data. This is of essence especially in turbulent economic and business environments, such as the ones faced by most EU countries today. The development of our instrument is discussed in the ‘Methods’ section.

In order to test our instrument and the suitability of the identified statistical methods to support regional policy making, we benchmarked the innovativeness of commercial companies in two geographically distinct regions of Greece, with diverse innovational characteristics: the ones in Crete (region 1) and Western Macedonia (region 2). According to Nikolaidis and Bakouros (2009), the structural characteristics of Cretan and Western Macedonian business environment are particularly restrictive. The various problems of the companies in these regions stress mainly issues of business viability, while it is difficult to foster the innovation process. Taking this particular business environment into consideration was the main reason for the realization of the research process that we present.

The empirical research that has been conducted in these regions has resulted in the percentages presented in Table 1. These represent the percentage of sampled commercial companies in both Greek regions that have specific innovation character and characteristics, according to the relative questions of the distributed questionnaire.

Table 1 Percentages of commercial companies in both examined regions regarding specific characteristics

We notice that all percentages of W. Macedonia are greater than those of Crete, but this examination alone cannot answer the query of whether the differences between them are significant. Conducting Test 1 for all characteristics, we determine that the differences of percentages are significant in every case; therefore, we should reject H0: P1 = P2 and accept H1: P1 < P2 for all nine examined characteristics. Additionally, we can calculate the p values of all innovation characteristics (presented in Table 2 in increasing order). Again, considering every common value of α, e.g., 1%, 5%, etc., the null hypothesis H0: P1 = P2 should be rejected for every characteristic as it is always p < α.

Table 2 p values for Test 1

The interpretation of those statistical results is that as the commercial companies in W. Macedonia obtain higher scores, they are more innovative than the Cretan ones. It is very important to point out that Crete and W. Macedonia are two Greek regions with different innovation and entrepreneurship characteristics, despite the fact that they are both at the top of the list of Greek regions regarding gross domestic product (GDP). Related studies (Bakouros 20062009) clearly demonstrate that, on one hand, Crete's GDP heavily depended on agriculture and tourism, and there is no obvious reason for companies to act differently regarding innovational characteristics. On the other hand, W. Macedonia is the heart of electricity production in Greece (producing more than 70% of the total national electricity consumption), and its high GDP can be attributed to this fact. Therefore, the profile of this region's entrepreneurship is more industrial, and its dynamic service sector presents a greater need for adaptation of new technologies. Evidently, those trends also affect the commercial companies under study.

Following the procedure presented analytically in the section ‘Research methodology’ regarding the conduct of Test 2 - namely the conduct of two one-tailed z tests - and recalling that in order to conclude that any two compared populations are equivalent, (1) both z tests must be statistically significant, and (2) the two z ζ values must have opposite signs; we have the results presented in Table 3. When ζ = 0.05, we notice that we cannot state equivalency (of percentages) of the two studied Greek regions for any innovation characteristic. In Table 4, we present the minimum ζ values that should be set by a researcher in order to arrive at an equivalency of the two populations of commercial companies regarding each one of the nine examined characteristics.

Table 3 Test 2 information on the nine examined characteristics and the populations of commercial companies
Table 4 Minimum ζ values that lead to equivalency of the two examined populations of commercial companies

Based on the latter statistical results, we conclude once again that the commercial companies in the two regions show significant differentiation between them in all studied characteristics. Nevertheless, we could separate them into two groups: the first one includes the characteristics for which equivalency of the two examined populations could be achieved more easily in the future, namely (in decreasing order of importance) cooperation with universities or research centers, usage of an email address, introduction of an organizational innovation/technology, and getting subsidies.

These are the characteristics for which relatively small minimum ζ values come out. The second group includes the rest of the characteristics, with larger minimum ζ values, for which equivalency between the examined populations is impossible to be considered and almost impossible to be achieved in the near future.

Τhe questionnaire that was distributed in both regions of Greece included a number of questions that permitted the determination of relative frequency distributions, i.e., the relative frequencies of commercial companies belonging to each of several non-overlapping classes. The latter was formed depending on the studied characteristic. The proper NPM for comparing the two, as indicated in ‘Research methodology’, is Test 3.

The relative frequency distributions of commercial companies in Crete and Western Macedonia, regarding the year of their foundation, are shown in Figure 1. The number of companies established in the two regions is increasing, and it is well indicated that in both Crete and W. Macedonia, the majority of companies were established in the late 1990s. The similar behavior of both samples of companies, regarding the examined characteristic, is proven statistically by conducting Test 3, which reveals the homogeneity of the two populations.

Figure 1

Relative frequency distributions of commercial companies regarding the year of their foundation.

What we also notice in Figure 1 is a small decrease in the establishment of new companies in the years 2000 to 2005. This behavior can be attributed to the general trend (at the national level) of less new companies established during the last decade, compared to what was happening in the past.

In Figure 2, we present the relative frequency distributions of companies in both studied regions regarding the number of their employees. We can see that the relative frequencies in both Crete and W. Macedonia decrease progressively as the total number of employees increases. Actually, most commercial companies in both areas have less than five employees.

Figure 2

Relative frequency distributions of commercial companies regarding the total number of their employees.

However, conducting Test 3, in this case, reveals that there is no homogeneity between the two populations of companies, and this is related to the fact that W. Macedonia is a more industrial-oriented region, compared to the agricultural- and tourist-oriented region of Crete. Therefore, there are rather larger companies in W. Macedonia regarding the number of their employees.

Next, we compare the two relative frequency distributions of the percentage of higher-education graduates employed by commercial companies in both Greek regions, which are presented in Figure 3. It is clear that as the percentage of graduates in each company increases, the relative frequency of companies decreases. Additionally, we notice that most companies in both Greek regions employ less than 10% of higher-education graduates. Regarding the specific characteristic of commercial companies, the homogeneity of the two populations can be proven statistically by conducting Test 3.

Figure 3

Relative frequency distributions of commercial companies regarding the percentage of higher-education graduates employed.

According to Bakouros (2006), Crete is one of the top three regions of Greece in the last 25 to 30 years regarding the establishments of higher educational institutions and the place where the biggest Greek research foundation, i.e., the Foundation of Research and Technology, has its headquarters. On the contrary, W. Macedonia (Bakouros 2009) only recently experienced the establishment of a university (much smaller than the ones in Crete) and a satellite research institute with a handful of researchers. Consequently, someone would expect Crete to demonstrate higher presence of graduates in companies, but this study does not prove that. This strange fact has to do with the low penetration of HEI and research centers in the local economy as it is very well pointed out and presented in Table 1: the percentage of commercial companies ‘cooperating with university or research centers’ is only 3.17% for Crete and a little higher, up to 9.58%, for W. Macedonia. Our empirical research proves that the need for cooperation between the academic and research world and the business community is in fact very high in both regions.

Focusing on the relative frequency distributions of the commercial companies in the studied regions regarding their target markets (Figure 4), we find out significant differences. More specifically, we realize that the local market is the most popular market in Crete, while in W. Macedonia, the most popular market is the international one. Not surprisingly, conducting Test 3 proves statistically the lack of homogeneity between the two examined populations.

Figure 4

Relative frequency distributions of commercial companies in the examined Greek regions regarding their target markets.

This lack of homogeneity was expected because commercial companies in W. Macedonia have to be more open to international markets in order to survive, while those in Crete rely mainly on tourism - both national and international - and up to now, they have not felt the pressure of being more aggressively market-related.

Finally, the study of the annual turnover and the respective relative frequency distributions of companies in both examined regions (Figure 5) reveals that the annual turnover of the majority of commercial companies fluctuates between 50,000 and 500,000 €. Despite the similarity of the two relative frequency distributions, Test 3 proves that there is no homogeneity between the two examined populations, and again, that is explained by the basic differences of the regional profiles (tourism-agricultural vs. industrial).

Figure 5

Relative frequency distributions of commercial companies in the examined Greek regions regarding their annual turnover.

Similar to what has been done for Test 1 in Table 2, we have calculated the p values of all characteristics regarding Test 3 in Table 5. Considering the usual values of α, e.g., 1%, 5%, etc., the null hypothesis, H0: each population presents (approximately) the same relative frequency at each one of the non-overlapping classes, should be rejected in three cases, namely regarding (1) the annual turnover, (2) the number of employees, and (3) the target markets.

Table 5 p values for all Test 3

Obviously, companies in both regions that had reached a necessary point of development so as to deal seriously with matters of innovation, strategic management, and organization were quite a few. On the contrary, most companies were simply interested in sales, logistics, and investments in fixed assets.


Our effort in this paper has been to develop an easy-to-use research tool and provide the appropriate methodology guidelines through presentation and application of statistical tests in order to support regional decision making process. Our methodology was applied in two regions with different innovational characteristics in Greece. The results indicate that commercial companies in Western Macedonia and Crete present similar behavior regarding the year of their foundation and the limited percentage of higher-education graduates they employ. On the contrary, the industrial region of Western Macedonia is more advanced in innovation and entrepreneurship issues in comparison to Crete, not only vis-à-vis the annual turnover, the number of employees, and the target markets but also regarding the rest of the characteristics examined in this research. More specifically, Western Macedonia has a greater number of firms using personal computers, having access to the internet, an email address, and a certified quality management system. Moreover, a larger amount of firms has introduced a new product or service, or an organizational innovation or technology and intend to continue to do so in the future. They also tend to more closely cooperate with universities or research centers and get subsidies.

The information above can help regional innovation policy makers examine the characteristics of specific regions and apply targeted measures that can lead to results that are more beneficial for each specific region. Alternatively, national policy bodies can examine differences between regions and support ones with limited innovative performance since, as seen in recent studies, public R&D funding was shown to increase the possibility of firms to become more innovative (Heimonen 2012). Additionally, as a supportive policy measure, increasing the accessibility of information sources can increase regional innovation capacity (Miika and Littunen 2010). Finally, this can help policy makers move away from application of general innovation policy measures and develop specific and suitable strategies for each region that can reveal its significant strengths and opportunities and lead to smart specialization strategies (Foray et al. 2009).

The discovery process presented previously can help regions that are not key players in major scientific and technological fields to develop a focused differentiation strategy and position themselves in the knowledge economy in a unique but strong position. However, the latter requires undertaking the identification of R&D and innovation activities which can be best developed in that region in comparison to other ones. It is of essence to do so, based on data collected rapidly and concerning recent and short periods, in order to be relevant and give the policy makers an opportunity to examine the progress of policy application.

Although our research is based on commonly used NPM, limited research has been enacted on using integrated statistical analysis that could be used for comparisons between different regions and support the work of regional policy making. Our effort, though not free of limitations, tries to create a pathway towards this goal and become the first step in order to develop and evaluate the effectiveness of a simple exploratory analysis, which can be used easily in practice in order to compare innovation and entrepreneurship characteristics between two regions.

Limitations and future research

Our study, being an initial effort to develop a simple rapid tool and provide the methodological background to measure and compare regional enterprise innovation, suffers from limitations. These include the limited number of innovation characteristics examined and the applicability of our research tool and methodologies on a single pair of regions. However, the scope of this research, as has been stated already, is to discover the potential usage of measurement and analysis of entrepreneurial regional innovation through nonparametric statistical techniques.

Future research is encouraged to study in more depth the individual characteristics of innovation and entrepreneurship systems and generate a ‘synthetic’ indicator based on more advanced statistical techniques that could serve as a reliable mirror of any innovative and successful enterprise. This synthetic indicator should be determined following an innovative, multi-level statistical approach based on collection and analysis of case study analyses and will depend, among others, on the market, the technology, and the operating environment of the individual enterprises. The synthesis could be accomplished using multi-criteria decision analysis and perhaps fuzzy logic, while it should also define critical success values by determining the best, medium, and worst practice cases. Overall, the result of such research, apart from all the used statistical techniques, should be very useful in evaluating the behavior of local/regional enterprises. Moreover, it should be useful to regional and national authorities for developing appropriate policies and practices, which promote real innovation and self-sustainable entrepreneurship.


Research methodology

As discussed in the ‘Results and discussion’ section, a questionnaire of limited length and sophistication had to be developed and additionally should have been distributed in a cost-effective and rapid manner. To this end, we designed a research tool that could be easily completed by a large number of firms regardless of their size and type. Its content was inspired by the questionnaire of the Fourth Community Innovation Survey (CIS IV), as well as by the questionnaires used in relevant surveys and projects that took place in the past. In particular, we designed a simple one-page questionnaire including 22 topics referring to the companies' profile and their fundamental characteristics, together with the data, the indicators, and the requirements of their structural or technological innovation. The topics were defined on the basis of our interest on the age, the principal activity, and the composition/quality of every company's human resources, together with more details for the level of computerization of its operation, product and process quality control, introduction or adaptation of new products, new technologies, and structural innovations. In parallel, collaboration with academic or research organizations was to an equal interest with the company's involvement in funding (internal or/and external) innovation activities. Finally, mapping of the company's markets was at a high interest. The questionnaire is presented in Table 6.

Table 6 Questionnaire of the telephone survey

In order to gather data rapidly and from a large number of firms, we conducted a telephone-based survey in both regions. The methodology that was used to determine (1) the statistical populations of companies participating in the survey, as well as (2) the samples that were examined during the telephone research, followed two steps. First, all commercial companies of both regions were traced (mainly through chambers and telephone directory services) and recorded. A random sample of these companies, corresponding to approximately 30% of the total number of companies, was determined and conducted to participate on the research.

Our emphasis has been mainly on the analysis of the data gathered in order to produce valid and useable results that could be an input to regional policy decision making. In our empirical research, the majority of the data collected through the questionnaires that were distributed to companies in both regions had been nominal/categoricala. More specifically, the questionnaire included a lot of binary questions in which the respondent companies were asked to give a ‘yes’ or ‘no’ answer. Consequently, the use of nonparametric statistical methods for the data analysis has been deemed necessary.

Nonparametric inferential statistical methods (NPM) are so called because the assumptions underlying their use are ‘fewer and weaker than those associated with parametric tests’ (Siegel and Castellan 1988, p. 34). That means that nonparametric methods require few if any assumptions about the shapes of the underlying population distributions. Due to their dependence on fewer assumptions, NPM are more robust, and their applicability is far wider than the corresponding parametric methods.

Established approaches of nonparametric statistical analysis are presented in remarkable books such as that of Siegel (1956), Siegel and Castellan (1988), Conover (1999), and Gibbons and Chakraborti (2003), as well as that of Sheskin (2007), Wasserman (2007), and Corder and Foreman (2009), where a comprehensive coverage of newly developed methods is provided.

Τhe measurement of companies' innovativeness which is presented in this paper has been based on properly chosen techniques of statistical analysis for the specific type of data collected from sample companies. In particular, the following techniques were considered:

  • The hypothesis-testing z test for difference between two independent percentages (Test 1)

  • The Westlake-Schuirmann test of equivalence of two independent percentages (Test 2)

  • The chi-square test for homogeneity of two populations (Test 3)

It is well known that in Test 1 there are two mutually exclusive and exhaustive statistical hypotheses, i.e., the null (H0) and the alternative (H A ) ones, which must cover all possible outcomes in a trial. In general, H A represents what a researcher seeks to prove, and H0 represents all remaining possibilities and is assumed true until proven otherwise. However, failing to reject H0 does not prove its validity. Therefore, sometimes a researcher needs to resort to a more in-depth statistical analysis such as the Westlake-Schuirmann test of equivalence of two independent percentages (Test 2).

More specifically, in every survey that is similar to the one presented in this research, through the use of Test 1, a researcher usually wants to test whether the difference between two ‘sampling’ percentages (proportions) is significant. The test procedure described below is appropriate when simple random sampling is used and the samples are large and independent. Consider as P1 the percentage of companies in a region that has a specific characteristic (e.g., has access to the internet), P2 the respective percentage of companies in another region, while d is the difference between the two population percentages. Assume that a researcher has reasons to believe that P1 < P2 + d. In order to examine this belief, he/she should carry out Test 1 by first stating the following hypotheses:

H 0 : P 1 - P 2 d or , equivalently , P 1 P 2 + d H 1 : P 1 - P 2 < d or , equivalently , P 1 < P 2 + d

The Test 1 statistic that is used is defined by the following equation:

Z = P ^ 1 - P ^ 2 P ^ 1 - P ^ 1 n 1 + 1 n 2

where n1 (n2) represents the size of the sample taken from population 1 (2), P ^ 1 ( P ^ 2 ) represents the percentage of sample 1 (2), and P ^ = n 1 P ^ 1 + n 2 P ^ 2 n 1 + n 2 is the pooled sample percentage. Choosing the significance level α and determining the p valueb (p), H0 is rejected whenever pα.

For instance, in our survey it is n1 = 945 (commercial companies in region A), n2 = 167 (commercial companies in region B), P ^ 1 = 40.53%, and P ^ 2 = 60.48%; consequently, the values of P ^ and p can be calculated as 43.53% and 0.0001%, respectively. This means that for every usual value of α, e.g., 1%, 5%, etc., H0: P1 = P2 should be rejected. Therefore, the conclusion that is derived from the hypothesis-testing procedure (HTP) is that the percentage of commercial companies in region A that have access to the internet, P1, is lower than the respective percentage P2 in region B, i.e., P1 < P2.

Usually, whenever Test 1 reaches the conclusion of non-statistically significant difference in percentages, this simply means that the current evidence is not strong enough to persuade the researcher that the two examined populations differ. However, that is not the same as saying that the two populations are equivalent. Moreover, sometimes a researcher's goal is not to prove that a statistically significant difference exists between the two studied population statistics, but that any existing difference is not important. Consequently, in contrast to the classical HTP, according to Test 2, H1 states that the examined statistics (e.g., means, variances, proportions, etc.) of two populations are in fact equivalent, while H0 states that there is a difference between them. Since it is not mathematically feasible to establish an alternative hypothesis, which would state the exact equality of statistics, whenever Test 2 is conducted, a parameter ζ is employed to reflect a maximum difference of statistics which would be tolerated in order to conclude that those statistics are equivalent. Any difference in statistics less than the absolute value of ζ would be viewed as insignificant, and thus, H0 should be rejected.

Based on the previous example, a researcher is willing to declare equivalence of the two regions regarding the percentage of companies that have this characteristic if the absolute difference between the percentages is less than ζ. To examine this belief, the researcher should rely on the following hypotheses:

H 0 : P 1 - P 2 ζ or , more analytically P 1 - P 2 ζ or P 1 - P 2 - ζ H 1 : P 1 - P 2 < ζ or , more analytically P 1 - P 2 < ζ or P 1 - P 2 > - ζ

while the used Test 2 statistic is given by the following equation:

Z ζ = P ^ 1 - P ^ 2 - ζ P ^ 1 - P ^ 1 n 1 + 1 n 2 .

Test 2 requires the conduct of two one-tailed z tests. In order for a researcher to conclude that the two populations are equivalent, (1) both z tests must be statistically significant, and (2) the two z ζ values must have opposite signs.

In our survey, let us consider that we are willing to declare equivalence of the two examined regions regarding the percentage of companies that have access to the internet if the absolute difference between the two population percentages is less than 5%. In this case, H0 and H1 are the following:

H 0 : P 1 - P 2 0.05 or P 1 - P 2 0.05 H 1 : P 1 - P 2 < 0.05 or P 1 - P 2 > 0.05 .

Initially, let us consider the first pair of H0 and H1, i.e., H0: P1 - P2 ≥ 0.05 and H1: P1 - P2 < 0.05. To reject H0 in this case, it should be P ^ 1 - P ^ 2 + z 1 - α P ^ 1 - P ^ 1 n 1 + 1 n 2 < ζ z ζ  < - z1 - α. Using Equation 4 and the values of n1, n2, P ^ 1 , and P ^ 2 determined previously, the result is zζ = 0.05 = - 5.99. Setting α = 1% (5%), we have the tabled critical one-tailed value - z1 - α = 0.99 = - 2.326 (- z1 - α = 0.95 = - 1.645). Since zζ = 0.05 = - 5.99 < - z α for α = 1% (5%), the H0, which designates nonequivalence of the percentages of companies P1 and P2 that have access to the internet, should be rejected.

Similarly, we consider the second pair of hypotheses, i.e., H0: P1 - P2 ≤ -0.05 and H1: P1 - P2 > -0.05. In this case, to reject H0, it should be ζ < P ^ 1 - P ^ 2 - z 1 - α P ^ 1 - P ^ 1 n 1 + 1 n 2 z 1 - α < z ζ . From Equation 4, we have that zζ = 0.05 = - 3.59 (as now ζ = -0.05). Setting α = 1% (5%), we have the following tabled critical one-tailed value z1 - α = 0.99 = 2.326 (z1 - α = 0.95 = 1.645). Since zζ = 0.05 = - 3.59 < z1 - α for α = 1% (5%), H0 cannot be rejected. Concluding, by noticing that only one of the two one-tailed z tests is statistically significant, while the two calculated z ζ values do not have opposite signs, we cannot consider that the two populations of companies are equivalent regarding their access to the internet.

Finally, an alternative statistical technique for processing the information/data collected from a sample is the chi-square test for homogeneity of two populations (Test 3). It can be used to determine whether two (or more) populations are similar/homogeneous in a characteristic. In our case, we want to find out whether the relative frequencies of companies having a specific ‘qualitative’ characteristic are distributed identically across the two examined populations. The general form of Test 3 is applied to a single categorical variable, using data from r ≥ 2 different populations. It is used to determine whether relative frequencies (or frequency counts) are distributed identically across those different populations. The test procedure described subsequently is appropriate when the following conditions are met: (1) the sampling method is simple random sampling, (2) each population is at least ten times as big as its respective sample, and (3) if frequency counts are displayed in a two-way contingency table; the expected frequency count for each cell of the table is at least 5.

In our study, data are collected from r = 2 populations, while it is supposed that the categorical variable has c levels. Based on that, H0 and H1 can be determined as follows:

H0: each population has the same percentage of observations at each of the c levels of the categorical variable.

H1: there is at least one level of the categorical variable, for which the two populations do not have the same percentage of observations.

Overall, to examine relationships between categorical variables, a contingency table constitutes a useful tool. Using sample data from such a table, a researcher can find the degrees of freedom and the expected frequency counts, as well as determine the value of the test statistic and the p value associated with the latter. More specifically, the degrees of freedom (DF) are defined as:

DF = r - 1 c - 1

The expected frequency counts Er,c are computed separately for each population r at each level c of the categorical variable according to the following formula:

E r , c = n r n c n

where n r is the total number of observations from population r, n c is the total number of observations at level c of the categorical variable, and n is the total sample size. The test statistic that is used in this case is a chi-square random variable defined by the following equation:

X 2 = r c O r , c - E r , c 2 E r , c

where Or,c is the observed frequency count in population r for level c of the categorical variable. H0 should be rejected at the significance level α every time the value of the test statistic determined in Equation 7 is greater than X 1 - α , DF 2 , i.e., whenever X 2 > X 1 - α , DF 2 .

Consider the study of the annual turnover and the respective contingency table of commercial companies in regions A and B (Table 7). In this case, we have r = 2, c = 5, and DF = 4, while Or,c and Er,c values are calculated and presented in Table 8. It can be easily determined from Equation 7 that X2 = 16.1603, and since X 1 - α , DF 2 = 9.4877 (13.2767) for α = 0.05 (0.01), then H0 should be rejected, i.e., the two studied populations are not equivalent.

Table 7 Contingency table for the study of the annual turnover of commercial companies in regions A and B
Table 8 O r , c and E r , c for the study of the annual turnover of commercial companies in regions A and B


aNominal/categorical data: the observations are classified into categories so that the data set consists of frequency counts for the various categories.

bp value is the probability of obtaining a result at least as extreme as the one that was actually observed given that H0 is true.

Authors' information

YN is an assistant professor at the Department of Technology Management, University of Macedonia, Greece. He is a mechanical engineer and earned his PhD from the Department of Mechanical Engineering - Polytechnic School, Aristotle University of Thessaloniki, in 2001. His research interests include quality management and statistical quality control, and also reverse logistics and innovation. KF holds the position of innovation for new products and services at the Department of Technology Management in the University of Macedonia, Greece. He holds a PhD in Strategic Management and an MSc in Decision Sciences specialized in e-Business from Athens University of Economics and Business. He has extensive research experience in EU national and private funded research programs and more than 15 years of working experience in business advisory for private and public sector organizations. His main research interests lie in the areas of strategic and innovation management and e-Business and innovative business models. EGC is a full professor of science, technology, innovation, and entrepreneurship as well as a co-founder and co-director of the Global and Entrepreneurial Finance Research Institute (GEFRI) and director of Research on Science, Technology, Innovation and Entrepreneurship, European Union Research Center, (EURC) at the School of Business of the George Washington University in Washington, DC. Dr. Carayannis' teaching and research activities focus on the areas of strategic government - university-industry R&D partnerships, technology roadmapping, technology transfer and commercialization, international science and technology policy, technological entrepreneurship, and regional economic development.


  1. Adams R, Bessant J, Phelps R: Innovation management measurement: a review. International Journal of Management Reviews 2006,8(1):21–47. 10.1111/j.1468-2370.2006.00119.x

    Article  Google Scholar 

  2. Archibugi D, Denni M, Filippetti A: The technological capabilities of nations: the state of the art of synthetic indicators. Technological Forecasting and Social Change 2009, 76: 917–931. 10.1016/j.techfore.2009.01.002

    Article  Google Scholar 

  3. Bakouros YL: Innovating in Crete. University of Western Macedonia: MATERLab Publication; 2006.

    Google Scholar 

  4. Bakouros YL: Regional innovation pole of Western Macedonia. MATERLab Final Report: University of Western Macedonia; 2009.

    Google Scholar 

  5. Barclay I: The new product development process: part 2. Improving the process of new product development. R&D Management 1992, 4: 307–317.

    Article  Google Scholar 

  6. Cebon P, Newton P: Innovation in firms: towards a framework for indicator development. Melbourne Business School: Working Paper; 1999.

    Google Scholar 

  7. Chu MT, Khosla R: Index evaluations and business strategies on communities of practice. Expert systems with applications 2009,36(2):1549–1558. 10.1016/j.eswa.2007.11.053

    Article  Google Scholar 

  8. Conover WJ: Practical nonparametric statistics. 3rd edition. New York: Wiley; 1999.

    Google Scholar 

  9. Cooper RG, Kleinschmidt EJ: The impact of product innovativeness on performance. Journal of Product Innovation Management 1991, 8: 240–251. 10.1016/0737-6782(91)90046-2

    Article  Google Scholar 

  10. Corder GW, Foreman DI: Nonparametric statistics for non-statisticians: a step-by-step approach. New Jersey: Wiley; 2009.

    Google Scholar 

  11. Cordero R: The measurement of innovation performance in the firm: an overview. Research Policy 1990, 19: 185–192. 10.1016/0048-7333(90)90048-B

    Article  Google Scholar 

  12. De Jong JPJ, Freel M: Absorptive capacity and the reach of collaboration in high technology small firms. Research Policy 2010,39(1):47–54. 10.1016/j.respol.2009.10.003

    Article  Google Scholar 

  13. Eom B-Y, Lee K: Determinants of industry–academy linkages and, their impact on firm performance: the case of Korea as a latecomer in knowledge industrialization. Research Policy 2010,39(5):625–639. 10.1016/j.respol.2010.01.015

    Article  Google Scholar 

  14. European Innovation Scoreboard EI: Comparative analysis of innovation performance. Belgium: Pro Inno Europe – Inno Metrics; 2008.

    Google Scholar 

  15. Foray D, David P, Hall B: Smart specialisation - the concept. Knowledge Economists Policy Brief no. 9. European Union. 2009.

    Google Scholar 

  16. Guan J, Yam R, Tang E, Lau A: Innovation strategy and performance during economic transition: evidences in Beijing, China. Research Policy 2009,38(5):802–812. 10.1016/j.respol.2008.12.009

    Article  Google Scholar 

  17. Geisler E: The metrics of science and technology. Westport: Quorum Books; 2000.

    Google Scholar 

  18. Gibbons JD, Chakraborti S: Nonparametric statistical inference. 4th edition. New York: Marcel Dekker; 2003.

    Google Scholar 

  19. Grupp H: Foundations of the economics of innovation – theory, measurement and practice. Cheltenham: Edward Elgar; 1998.

    Google Scholar 

  20. Guillaume R, Doloreux D: Production systems and innovation in ‘satellite’ regions: lessons from a comparison between Mechanic Valley (France) and Beauce (Québec). International Journal of Urban and Regional Research 2011, 35: 1133–1153. 10.1111/j.1468-2427.2011.01044.x

    Article  Google Scholar 

  21. Gunnarsson-Östling U, Höjer M: Scenario planning for sustainability in Stockholm, Sweden: environmental justice considerations. International Journal of Urban and Regional Research 2011, 35: 1048–1067.

    Google Scholar 

  22. Heimonen T: What are the factors that affect innovation in growing SMEs? European Journal of Innovation Management 2012,15(1):122–144. 10.1108/14601061211192861

    Article  Google Scholar 

  23. Herrera L, Nieto M: The national innovation policy effect according to firm location. Technovation 2008,28(8):540–550. 10.1016/j.technovation.2008.02.009

    Article  Google Scholar 

  24. Huff AS: Mapping strategic thought. New York: Wiley; 1990.

    Google Scholar 

  25. Hult GTM, Hurley R, Kingth G: Innovativeness: its antecedents and impact in business performance. Industrial Marketing Management 2004, 33: 429–438. 10.1016/j.indmarman.2003.08.015

    Article  Google Scholar 

  26. Kim B, Oh H: An effective R&D performance measurement system: survey of Korean R&D researchers. Omega – International Journal of Management Science 2002, 30: 19–31. 10.1016/S0305-0483(01)00049-4

    Article  Google Scholar 

  27. Liao SH, Fei WC, Liu CT: Relationships between knowledge inertia, organizational learning and organization innovation. Technovation 2008, 28: 183–195. 10.1016/j.technovation.2007.11.005

    Article  Google Scholar 

  28. Mainwaring L, Moore NJ, Murphy PD: A regional comparison of enterprise patent holdings: a study of British and Irish data. Research Policy 2007,36(10):1655–1665. 10.1016/j.respol.2007.09.005

    Article  Google Scholar 

  29. Nada N, Kholeif M, Elbadawy A, Yanik T: An integrated innovation management framework. In Proceedings of the 5th International Conference on Innovation and Entrepreneurship. Izmir: Izmir University; 2010.

    Google Scholar 

  30. Nardo M, Saisana M, Saltelli A, Tarantola S, Hoffman A, Giovannini E: Handbook on constructing composite indicators: methodology and user guide. Paris: OECD; 2008.

    Google Scholar 

  31. Nikolaidis Y, Bakouros Y: Innovation penetration into a region with specific features: the case of Crete, Greece. International Journal of Entrepreneurship and Innovation Management 2009, 9: 118–138. 10.1504/IJEIM.2009.023848

    Article  Google Scholar 

  32. Phelps B: Smart business metrics: measure what really counts and manage what makes the difference. London: FT-Prentice Hall; 2004.

    Google Scholar 

  33. Raymond L, St-Pierre J: R&D as a determinant of innovation in manufacturing SMEs: an attempt at empirical clarification. Technovation 2010,30(1):48–56. 10.1016/j.technovation.2009.05.005

    Article  Google Scholar 

  34. Sánchez A, Lago A, Ferràs X, Ribera J: Innovation management practices, strategic adaptation, and business results: evidence from the electronics industry. Journal of Technology Management & Innovation 2011,6(2):14–39. 10.4067/S0718-27242011000200002

    Article  Google Scholar 

  35. Sheskin DJ: Handbook of parametric and nonparametric statistical procedures. 4th edition. New York: Chapman & Hall; 2007.

    Google Scholar 

  36. Siegel S: Nonparametric statistics for the behavioral sciences. New York: McGraw-Hill; 1956.

    Google Scholar 

  37. Siegel S, Castellan NJ: Nonparametric statistics for the behavioral sciences. New York: McGraw-Hill; 1988.

    Google Scholar 

  38. Sorescu AB, Chandy RK, Prabhu JC: Sources and financial consequences of radical innovation: insights from pharmaceuticals. Journal of Marketing 2003, 67: 82–102. 10.1509/jmkg.

    Article  Google Scholar 

  39. Tan B, Lee C-K, Chiu J-Z: The impact of organizational culture and learning on innovation performance. International Journal of Innovation and Learning 2008,5(4):413–428. 10.1504/IJIL.2008.017561

    Article  Google Scholar 

  40. Tidd J: Innovation management in context: environment organization, and performance. International Journal of Management Review 2001, 3: 169–183. 10.1111/1468-2370.00062

    Article  Google Scholar 

  41. Miika V, Littunen H: Types of innovation, sources of information and performance in entrepreneurial SMEs. European Journal of Innovation Management 2010,13(2):128–154. 10.1108/14601061011040221

    Article  Google Scholar 

  42. Wasserman L: All of nonparametric statistics. Berlin: Springer; 2007.

    Google Scholar 

Download references


The authors would like to thank Professor Yiannis Bakouros for his contribution towards this article.

Author information



Corresponding author

Correspondence to Yiannis Nikolaidis.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

YN carried out a significant part of the presentation of the previous research, developed the research methodology, conducted its application to specific regions, and performed the statistical analysis. He also drafted the manuscript. KF participated in the sequence alignment, carried out a part of the presentation of the previous research, determined the limitations and the future research, and most importantly designed the aims of the manuscript. EG's contribution focused on the determination of the scope of the manuscript. All authors read and approved the final manuscript.

Authors’ original submitted files for images

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Nikolaidis, Y., Fouskas, K. & Carayannis, E.G. Assisting regional policy by rapidly comparing enterprise innovation between regions. J Innov Entrep 2, 16 (2013).

Download citation


  • Innovation
  • Regional development
  • Regional policy
  • Innovation measurement
  • Nonparametric statistical methods