Measurement for Policy

Why is measurement important for policy?
Appropriate measurement is critical for policy to support innovation (OECD 2010a, 2010b) since it may help policy makers in accomplishing the following:
 
  • Assessing the contribution of innovation to achieve social and economic objectives.
  • Understanding the determinants of and obstacles to innovation to design policies with higher chances of success.
  • Evaluating the effectiveness of different policy approaches, and consequently adapting current policies or designing new ones.
  • Benchmarking innovation performance and conditions for innovation to those of other countries.
What types of measurement systems are needed for innovation policy?
  • Determinating factors for innovation. The measurement system should adopt a broad approach to innovation determinants. Innovation is affected by a wide range of factors at multiple levels of analysis (e.g. those determined at the firm, industry, region and country levels). At the firm level, for instance, determinants include not only R&D but multiple other complementary intangible assets, such as software, human capital and new organizational structures. It is critical to go beyond science, technology and innovation indicators to take into broader conditions for innovation (e.g. education, entrepreneurship, access to finance, labour market).
  • Conceptual analyses. Measurements should go beyond targets and aggregates to an analysis level that will help understand why and how innovation happens in firms. Innovation surveys can increase knowledge about why and how innovation happens in firms by collecting information about innovation strategies, reasons for investing in innovation, focusing or combining certain types and modes of innovation, as well as quantitative data on sales from product innovations and spending on a range of assets beyond R&D.
  • Role of government. The measurement system should address the role of government, including central and local government and various agencies, in fostering innovation. The nature, direction and intensity of policy actions for innovation at national and regional levels need to be measured in order to better understand the relevance of policies in different innovation system contexts.
  • Capture of knowledge interactions. The production of new knowledge is often a collective process involving individuals and organizations within networks. These networks typically cluster in certain geographic location or around certain institutions. In addition, technologies often draw on interdisciplinary research and can be used across a broad range of industries. Such interactions across actors, locations, and technologies need to be tracked as part of the innovation measurement framework.
  • Measurement of social impacts. Beyond economic goals, this analysis should measure the social impacts of innovation—evaluating not only the contribution of innovation to economic performance, but also its impact on well-being and its contributions to achieving social goals.
What are the key statistical sources for measuring innovation?
Inputs for measurement include various statistical sources, such as the following:
Industry (STAN, IO) data
The OECD provides multiple databases of internationally comparable statistics that include indicators on the performance of industries (Measuring Industrial Performance):
 
The STructural ANalysis Database provides a comprehensive tool for analysing industrial performance at a detailed level of activity across countries. It includes annual measures of R&D expenditures in industry, as well as measures of labour input, investment, and international trade. The STAN database is primarily based on annual national accounts by activity tables and uses data from other sources, such as national industrial surveys/censuses, to estimate any missing detail.
The OECD also provides tables that illustrate flows between the sales and purchases (final and intermediate) of industry outputs, reflecting the relationships between producers and consumers within an economy. The latest set of OECD Input-Output tables includes matrices of inter-industrial flows of goods and services transactions for almost all OECD countries and 15 non-member countries, covering the years 1995, 2000 and 2005 or nearest years.
 
These databases are used to study framework conditions that impact innovation (e.g. labour market characteristics) and innovation input (e.g. R&D expenditure). They are also useful to compute indicators of innovative activities, such as the R&D intensity by industry (i.e. the ratio of R&D expenditures to value added or to production), that provide insight into the relative importance of R&D across industries.
 
Firm level microdata
Microdata-based indicators are useful to reflect differences in innovation performance across firms. These indicators can characterise firms by size, by industry, and other relevant subgroups of firms. Moreover, firm-level microdata are critical for analysing determinants of firms’ innovation behaviour, a key issue for designing effective innovation policies.
 
The OECD Innovation Microdata Project launched in 2006 examined a range of issues related to innovation and firm performance, combining a common methodology in 20 countries with the work undertaken by researchers with access to their own country’s micro-data (OECD, 2009a). The themes selected for econometric analysis included the determinants of innovation and the impact on productivity; modes of innovation, including non-technological innovation; and the incentive effect of IPR on innovation.
Labour force survey and household microdata
A labour force survey can provide useful data to analyze the effects of labour force characteristics (e.g. unit labour costs, employment by industry, employees’ mobility) on business innovation. In addition, household microdata (e.g. data on ICT usage) can be used to track the adoption of innovation (e.g. the intensity of use and practices), a critical factor, particularly for assessing the welfare effects of innovations.
R&D statistics
The Frascati Manual (OECD, 2002) provides internationally accepted definitions of R&D and classifications of its component activities, contributing thereby to intergovernmental discussions on “best practices” for science and technology policies. The manual has become a standard for R&D surveys worldwide.
 
The OECD Science, Technology and R&D Statistics combine Research & Development Statistics (RDS) with the Main Science and Technology Indicators (MSTI):
  • MSTI database (link to the database) provides a set of indicators (link to the document) that reflect the level and structure of the efforts undertaken by OECD Member countries and seven non-member economies (Argentina, China, Romania, Russian Federation, Singapore, South Africa, Chinese Taipei) in the field of science and technology from 1981 on. These data include final or provisional results as well as forecasts established by government authorities. The indicators cover the resources devoted to research and development, patent families, technology balance of payments and international trade in R&D-intensive industries.
  • OECD Research and Development Statistics (RDS – link to the database). This database is based on the data reported to OECD and Eurostat in the framework of the joint OECD/Eurostat international data collection on resources devoted to R&D. The statistical series start from 1981 and cover the following fields with various breakdowns: R&D expenditure (e.g. expenditure on research and experimental development by sector of performance and source of funds), R&D personnel (e.g. R&D personnel by sector of employment and qualification) and government R&D budgets by socio economic objective (GBAORD).
Innovation data
The Oslo Manual (OECD/Eurostat, 2005) provides a framework for countries to develop internationally comparable innovation surveys. It is the foremost international source of guidelines for the collection and use of data on innovation activities in industry. Since the manual was first developed by OECD and Eurostat in 1992, it has been updated twice, the third edition published in 2005. While previous editions emphasised technological product and process (TPP) innovation, the latest edition extends the scope of the surveys to marketing and organisational innovation. It identifies four types of innovation: product, process, marketing and organisational innovations, and places new emphasis on the role of linkages (including collaboration) in innovation.
Patents and IPR
Patents can be considered an intermediate step between R&D (upstream) and innovation (the invention is used downstream in economic processes). A patent can be sought at different stages of the R&D process, notably in the case of incremental or cumulative inventions. Patents can be seen not only as an output of R&D but also an input to innovation and thus as both inputs and outputs in the invention process. This intermediate character makes patent data a useful bridge between R&D data and innovation data (both of which are collected through business surveys).
 
Patent data can support the analysis of other dimensions of innovation that are of policy interest, such as the characteristics of innovative activities (see Characteristics of innovative activities) such as including analysis of the performance of firms, regions and countries (see Patent data - The performance of firms, regions and countries), p tracking of linkages in the S&T system,  understanding the evolution of emerging technologies (see Patent data - Emerging technologies) as well as gaining perspectives on the role of universities in technological development (see Patent data - The role of universities in technological development), the performance and mobility of researchers (see Patent data - The performance and mobility of researchers), the geography of invention (see Patent data - The geography of invention) and knowledge diffusion and technological change (see Knowledge diffusion and technological change). Yet, patent-based indicators suffer from several weaknesses and, as a result, they should be designed and interpreted with care (see Aspects and availability of patent data). Other types of registered intellectual property rights, such as trademarks, design rights and utility models, can provide additional relevant information for innovation policy.
Bibliometrics
Bibliometrics is a set of methods to statistically analyze scientific and technological output. Bibliometrics can be used to measure the impact of specific articles, authors, and publications on scientific disciplines (e.g. by counting the number of citations), scientific linkages, and the extent of international co-operation (e.g. as evidenced by joint signatures). It is an important tool in evaluating research activities, laboratories and scientists, as well as the scientific specialisations and performance of countries (OECD, 1997).
What output formats provide measures related to innovation?
IPP.Stat
The IPP.stat will provide a collection of innovation-related indicators. It will allow users to interact with and visualise data, and aid them in gaining an appreciation of the relative positioning of their country.
 
Statistics and indicators by topic, activity, country and actor
OECD STI Outlook Country Profiles (OECD 2012, 2010) present the individual science, technology and innovation performance of OECD countries and some non-OECD countries, their national context and current major policy issues. Profiles are based on  the responses provided by countries to OECD policy questionnaires, as well as various additional OECD and non-OECD sources.

OECD Innovation Policy Platform. The various nodes of the OECD Innovation Policy Platform gather a set of indicators about key characteristics, actors, and determinants of innovation for each of the following thematic modules:
 
 
Scoreboard
The OECD Science, Technology and Industry (STI) Scoreboard is a biennial publication that provides a wide set of indicators of science, technology, globalisation and industrial performance in OECD and major non-OECD countries; it also presents major world trends in knowledge and innovation (OECD, 2011, 2009c, 2007).
References
  • OECD (2013), OECD Science, Technology and Industry Scoreboard 2013, OECD Publishing.
  • OECD (2012), “Country profiles”, in OECD Science, Technology and Industry Outlook 2012, OECD Publishing. doi: 10.1787/sti_outlook-2012-en
  • OECD (2011), OECD Science, Technology and Industry Scoreboard 2011, OECD Publishing. doi: 10.1787/sti_scoreboard-2011-en
  • OECD (2010a), "Improving Governance and Measurement", in The OECD Innovation Strategy: Getting a Head Start on Tomorrow, OECD Publishing. doi: 10.1787/9789264083479-9-en
  • OECD (2010b), Measuring Innovation: A New Perspective, OECD Publishing.doi: 10.1787/9789264059474-en
  • OECD (2010c), “Country profiles”, in OECD Science, Technology and Industry Outlook, OECD Publishing. doi: 10.1787/sti_outlook-2010-en
  • OECD (2009a), Innovation in Firms: A Microeconomic Perspective, OECD Publishing. doi: 10.1787/9789264056213-en
  • OECD (2009b), OECD Patent Statistics Manual, OECD Publishing, doi: 10.1787/9789264056442-en
  • OECD (2009c), OECD Science, Technology and Industry Scoreboard 2009, OECD Publishing. doi: 10.1787/sti_scoreboard-2009-en
  • OECD (2008), “Country profiles”, in OECD Science, Technology and Industry Outlook 2008, OECD Publishing. doi: 10.1787/sti_outlook-2008-en
  • OECD (2007), OECD Science, Technology and Industry Scoreboard 2007, OECD Publishing. doi: 10.1787/sti_scoreboard-2007-en
  • OECD (2002), The Measurement of Scientific and Technological Activities—Frascati Manual 2002: Proposed Standard Practice for Surveys on Research and Experimental Development, , Annex 7, paras. 21-25, OECD Publishing. doi: 10.1787/9789264199040-en
  • OECD/Statistical Office of the European Communities, Luxembourg/European Union (1997),Proposed Guidelines for Collecting and Interpreting Technological Innovation Data: Oslo Manual, The Measurement of Scientific and Technological Activities, OECD Publishing.doi: 10.1787/9789264192263-en
  • OECD/Eurostat (2005), The Measurement of Scientific and Technological Activities—Oslo Manual: Guidelines for Collecting and Interpreting Innovation Data, 3rd ed., OECD Publishing. doi: 10.1787/9789264013100-en
Links to databases and I/O tables
Image description here.

What Countries are Doing