A Joint Survey. The Danish approach to CIS and R&D in the Business Sector. November 2012. Nils Galberg Enoksen, Statistics Denmark

A Joint Survey The Danish approach to CIS and R&D in the Business Sector November 2012 Nils Galberg Enoksen, Statistics Denmark

A joint survey the Danish approach to CIS and R&D in the Business Sector Preface This paper contains the results of the study on the evaluation of the Danish experiences on the joint data compilation of the community Innovation Survey and the Research & Development statistics for the Business Enterprise sector. The report is written by Nils Galberg Enoksen, Statistics Denmark, and is part of work package 2, part 2 of grant agreement No. 50602.2011.002-2011.269. Since the reference year 2007, where Statistics Denmark took over the responsibility of compiling statistics on research and development and innovation, the R&D Survey for the Business Enterprise sector (Commission Regulation N 753/2004) and the CIS (Commission Regulation N 1450/2004) have been compiled as a joint survey. Furthermore the CIS is collected every year and therefore 4 complete surveys with both R&D and CIS-data from the years 2007-2010 have been available for the analysis and evaluation. Contents Preface... 1 Introduction to the Danish R&D and Innovation survey... 2 Background (History)... 2 Description of the joint survey model... 4 Legislation... 4 Time series... 4 Sample and Frame... 4 Data collection... 6 Contents of questionnaire and yearly variances... 7 Pro et Cons... 8 Analysis... 10 Data 2007-2010 (Part 1)... 10 Estimated time use...10 Combination of R&D and innovative enterprises...12 Review from respondents 2010 (Part 2)... 14 Time spent...14 Understanding the questions...16 Gather the information needed...16 Conclusions... 18 Road ahead...18 Annex 1 Overview of routing in the 2009 questionnaire... 19 Annex 2 The Danish 2009 questionnaire... 20 1

Introduction to the Danish R&D and Innovation survey Background (History) On 1 January 2008 Statistics Denmark took over the responsibility of compiling statistics on research and development and innovation. The responsibility was transferred on the basis of a government decision that was made in November 2007. The decision stated that the role of Statistics Denmark as a central authority for compiling statistics should be strengthened. Due to demands from national users it was agreed that Statistics Denmark should compile both the R&D and the innovation statistics on a yearly basis instead of the two year cadence as done by the vast majority of the member states. It was also agreed to integrate the R&D and innovations surveys into one single survey. Hence the R&D statistics and the Innovation Statistics for the business sector have been processed as one joint survey instead of two separate statistical areas since the reference year 2007. The merge of the two statistical areas is based on the fact that there is close relationship between the surveys both in terms of sampling frames and the conceptual contents. The populations for both surveys have a large intersection and also there are many similarities between the fields of interest of R&D and innovation. A steering committee monitored the project of taking over the statistics, which along with the business sector R&D and innovation surveys also included the statistics of R&D in the public sectors, the statistics of doctorate holders and the statistics on government budget appropriations (GBOARD). In the agreement of taking over the statistics is included that the innovation survey should be carried out yearly. The first year of data collection being 2007, this implied launching a full R&D survey for the business enterprise sector as well as an innovation survey. The steering committee discussed the possibilities of data collection and decided that the two data collections should be collected via one questionnaire, on the grounds that: Enterprises would experience the replying of one combined questionnaire less burdensome than two separate questionnaires, taking into account the fact that a substantial part of the survey population would be the same for the two surveys. As some questions concerning the expenses for R&D and innovation had to be included in both questionnaires for those enterprises, who were only part of one of the two surveys, this would imply either development of different questionnaires, or repetition of the same questions in the two surveys. The steering committee was of the opinion that Statistics Denmark would benefit from combining the surveys, as all questions on innovation expenses, including R&D, would be collected in the same questionnaire and therefore could provide a better quality: the same person filling out the questions would reduce the risk of reporting the same amounts twice, as both innovation and R&D expenses. Statistics Denmark was at that time beginning to implement electronic questionnaires, and the data collection conc. R&D and innovation should also include the possibility of electronic responses, implying the set-up of the system for these surveys with a substantial number of variables. The electronic questionnaire would allow for routing - that is, the respondents would only be presented to relevant questions. E.g. if a respondent did not have any R&D activities whatsoever, he or she would only have to respond to the introductory questions on R&D, and would automatically be lead to the innovation questions. 2

The combined R&D and innovations surveys is in Danish known as Forskning, udvikling og innovation abbreviated to the FUI-Statistics. As an English abbreviation R&D&I is used in this report. Among national users with substantial interest in the R&D&I statistics is the Danish Ministry of Science Technology and Higher Education 1, Ministry of Business and Growth 2 and, Ministry of Economic Affairs and the Interior 3. These actors and their specific demands have contributed to the design of the Danish statistics on research, development and innovation. 1 http://en.fivu.dk/ 2 http://www.evm.dk/english 3 http://english.oim.dk/ 3

Description of the joint survey model Due to the demands from national users The Danish approach to compiling the R&D&I statistics implies some deviations from the general approach such as this is described (recommended) by Eurostat/the European Commission. These deviations in terms of design of the questionnaire, frequency of the statistics and sample and frame populations are described in the following paragraphs. Legislation Data to the Danish R&D&I statistics are collected in accordance with section 8 of the Act on Statistics Denmark (Consolidated act No. 599 of June 22, 2000). Hence it is mandatory for all enterprises to report the demanded information about the firms R&D and innovation. Data are collected in accordance with Decision No 1608/2003/EC of the European Parliament and of the Council of 22 July 2003 concerning the production and development of Community statistics on science and technology and Commission Regulation (EC) No 753/2004 of 22 April 2004 implementing Decision No 1608/2003/EC of the European Parliament and of the Council as regards statistics on science and technology Time series Statistics Denmark has compiled the R&D&I statistics as a joint survey since the reference year 2007. The joint surveys have been processed every year since then with the latest published statistics from 2010 leaving a time series from 2007-2010. Sample and Frame As seen in Table 1 an average of 5 enterprises are contacted with request to report data on R&D&I. Due to non-responding units, reports from groups of enterprises, and entities allocated to the private-non-profit or public sectors, the number of responding units is approximately 4.400 pr. year representing a sampling frame of roughly 20 enterprises. All NACE industries are surveyed for all years although an exception is sections O, P, and Q, which are only periodically a part of the population The units from section O and P selected in 2007-2009 are mainly entities registered in the public- or privatenon-profit sector, but which have been considered a part of the Business Enterprise R&D statistics. From 2010 sections O and P are no longer surveyed. In Section Q only very few enterprises are operating under market conditions and as such contributing to the BES. Enterprises in section Q operating under market conditions who maintain R&D activities on a regular basis are part of the R&D&I sample and frame. A major player in section M is aggregation MB Scientific research and development of which all units are selected for the survey. 4

Version til korrektur 13/11-2012 Table 1. Sample and population. Distribution on NACE-sections (rev.2) 2007 2008 2009 2010 n (resp) N (pop) n (resp) N (pop) n (resp) N (pop) n (resp) N (pop) NACE Sections A Agriculture, forestry and fishing 13 38 10 27 12 33 12 15 B Mining and quarrying 22 38 30 66 32 63 31 49 C Manufacturing 1.215 4.618 1.085 5.385 1.014 5.051 1.219 4.735 D Electricity, gas, steam and air conditioning supply 22 59 28 140 49 132 34 79 E Water supply; sewerage, waste management and remediation activities 33 43 37 82 41 85 52 91 F Construction 150 377 97 396 81 312 105 226 G Wholesale and retail trade; repair of motor vehicles and motorcycles 938 5.720 1.050 5.907 835 5.724 808 4.481 H Transportation and storage 136 269 107 287 149 268 154 239 I Accommodation and food service activities 172 1.463 120 1.589 121 1.562 67 224 J Information and communication 549 2.358 916 2.713 793 2.688 594 3.084 K Financial and insurance activities 142 326 212 578 220 534 194 557 L Real estate activities 71 516 40 767 97 542 25 61 M Professional, scientific and technical activities 785 3.306 490 2.225 739 3.744 701 4.153 N Administrative and support service activities 238 1.026 163 1.316 299 1.332 283 1.403 O Public administration and defence compulsory social security 4 4 5 6 5 5.. P Education 3 11...... Q Human health and social work activities.. 4 6 4 4 3 4 R Arts, entertainment and recreation 4 31 14 431 23 345 17 23 S Other service activities 37 257 30 294 31 315 23 59 All NACE 4.534 20.460 4.438 22.215 4.545 22.740 4.322 19.483 R&D and Innovation statistics. Statistics Denmark 2007-2010 Looking at the size classes surveyed as seen in Table 2 all enterprises with more than 250 employees are selected. The share of selected units decreases with the size of the units. On average every second enterprises with 50-249 employees is selected from all sections. Enterprises with 1-9 and 10-49 employees are more randomly surveyed depending on the industry. The share of units selected within section M is strongly influenced by aggregation MB which by far accounts for most of the selected units in enterprises from 1-9 and 10-49 employees. 5

Version til korrektur 13/11-2012 Table 2. Selection probability by enterprise size class on NACE sections (rev. 2). 2010 Enterprise size - No. of employees 1-9 10-49 50-249 250-499 500-999 +1000 All sizes Percentage of reporting enterprises NACE Sections A Agriculture, forestry and fishing - - 77 100 100-80 B Mining and quarrying 36 64 100 100 100-63 C Manufacturing 12 19 59 100 100 100 26 D Electricity, gas, steam and air conditioning supply - 30 75 100 100 100 43 E Water supply; sewerage, waste management and remediation activities - 45 71 100 100-57 F Construction - - 40 100 100 100 46 G Wholesale and retail trade; repair of motor vehicles and motorcycles 9 14 45 100 100 100 18 H Transportation and storage - 100 56 100 100 100 64 I Accommodation and food service activities - 17 52 100 100-30 J Information and communication 14 22 53 100 100 100 19 K Financial and insurance activities 16 23 61 100 100 100 35 L Real estate activities - 23 60 100 - - 42 M Professional, scientific and technical activities 11 23 56 100 100 100 17 N Administrative and support service activities 10 16 51 100 100 100 20 Q Human health and social work activities - - 67 100 - - 75 R Arts, entertainment and recreation - - 70 100 100-74 S Other service activities - 22 91 100 100-39 All NACE 12 19 54 100 100 100 22 Data collection The Danish R&D&I data collection is mandatory, which results in very high response rates. In 2009 the response rate was 93 per cent. The statistics are compiled on the basis of questionnaires. In 2007 the primary (and only) media was a paper questionnaire. Since the reference year 2008 reports on web-based questionnaires has also been an option for reporting data. Since the introduction of web-based forms in 2008 the share of web-based reports has increased to approximately 75 per cent for the 2010 data collection. From the reference year 2012 the respondent s only media for reporting will be the e- questionnaire since data collection from web-based media will be mandatory. The e-questionnaire is dynamic meaning that the questions presented to the respondent are depending on the answer given to filtering questions. In specific terms this means that enterprises who in the filtering questions report no R&D activities are transferred directly to the part of the questionnaire containing questions on innovation activities. In this way respondents will not be forced to read or even relate to a series of questions, which are irrelevant in this context. This is of substantial significance, when dealing with a questionnaire as extensive as the Danish R&D&I. The routing and filter questions in the 2009 questionnaire are presented in Annex 1. 6

Contents of questionnaire and yearly variances The contents of the questionnaire vary from odd to even years. As seen in table 3 the extent of the questionnaire measured by the number of variables varies from 222 variables in even years to 390 variables in odd reference years. The questionnaire can be broken down to 4 overall element; R&D-, innovation-, voluntary - and general information modules. Table 3 Yearly variances on extent of questionnaire Even years Odd years Extent Variables Modules Extent Variables Modules General information 10 (5) 2 10 (5) 2 R&D 4 Reduced 36 (6) 5 Full 217 (6) 13 Innovation mandatory Full 163 (16) 12 Full 163 (16) 12 All mandatory 209 (27) 19 390 (27) 27 Innovation voluntary 5 13 (2) 2 - - - All Questionnaire 222 (29) 21 390 (27) 27 Note: The number of variables is listed as the total number of variables and the number of filter variables is shown in the parentheses. In table 3 the extent of variables are listed as the total number of variables followed by the number of filter variables in the parentheses. The filter questions make the minimum number of variables that a responding unit have to answer. The variation in filter questions only varies between 29 variables in even years and 27 in odd years. For all reference years the modules on general information and on innovation are mainly unchanged. The general information is based on 2 modules with basic information about the enterprise, e.g. is the enterprise a part of a group of enterprises, and a module conc. contact information. The 2 modules contain 10 variables of which 5 are from filter questions. The full innovation survey is collected every year. The innovation part consists of 12 modules of questions containing 163 variables of which 16 are filter variables and hence must be answered by all responding units. The innovation part has in even years been followed by modules with voluntary questions on innovation. In 2010 the subject was Design and innovation and Experiences and innovation. These voluntary questions are described in the paper Design and Experiences 6. The R&D part shows distinct differences from odd to even years, in accordance with the demands for data following from the Decision No 1608/2003/EC. In even years 6 filter questions must be answered potentially leading to 5 modules with a total of 36 variables. In odd years the 6 filter questions lead to 13 modules containing a total of 217 variables. As an example the full questionnaire for the 2009 survey is presented in annex 2. As described in the text above the Danish approach to compiling R&D and innovation statistics involve a great degree of change in the questionnaire from even to odd years. Due to the filtering in the questionnaire and the design of web application form the yearly variations are only noticeable for a minor part the population, namely the 4 Includes intramural and extramural R&D 5 Based on the 2010-questionairre 6 Design and Experiences Analysis Report by Helle Månsson, Statistics Denmark, 2012. 7

enterprises which tick yes to having intra- or extramural R&D activities. Changes are therefore not visible for the vast majority of respondents. Pro et Cons Both advantages and disadvantages have been identified in connection with working with the Danish R&D&I. The Danish approach to producing R&D&I statistics is based on the assumption that it implies advantages such as 1) reduction of the response burden for the responding units and 2) improving conditions for the production of and thus the quality of the statistics, including data collection and data processing. From the initial stages of the project of taking over the statistics the following pros and cons have been identified: Advantages A joint data collection results in less administrative burdens: o One questionnaire instead of 2-3 questionnaires o Reduction in the number of letters to enterprises o Fewer general questions asked One questionnaire per survey o Joint system of terminology o Easier to explain the differences and connections of the concept R&D and innovation in the same questionnaire One contact per enterprise o Fewer persons involved One sample and one frame o Less administration for the NSO o New combinations for R&D and Innovation statistics possible Larger sample reduces statistical uncertainty Improved data processing o More consistent data o Reduction in the process of validation Error detection is improved o Better conditions for identifying errors and misinterpretations in data Weaknesses Extensive questionnaire o The questionnaire is voluminous and contains concepts and terms which may not always be fully comprehensible or hard to interpret to the respondents Yearly alterations of the questionnaire are costly o Two different questionnaires for even and odd years o Possible bias due to changing of the questionnaires The overall response burden will increase hence to the sample for BES R&Dsurvey will be larger than the previous approach One contact per enterprise o The one contact may not necessarily be the correct contact. It can seem conflicting that the response burden is treated as a gain in respect to reducing the response burden AND at the con side as an increase of the burden. To explain this puzzling situation one must bear in mind that the direct result of a joint survey is reduction of the number of letters send to the selected enterprises from the 8

first letters of notification to reminders or for that matter phone reminders and enquiries from the NSO to the enterprise regarding the validation of the reported data. Another point treated both as a gain and a weakness in the R&D&I approach is the access to the enterprise via one contact. The contact may not necessarily be the person answering all the questions in the questionnaire, but could also be a key figure, who collects various information in various departments of the enterprise. However the situation where the contact is not the correct contact meaning that the questionnaire is filled in by a respondent with limited access to the necessary information. This should be considered a weakness but not directly because of the survey design, since this mistake easily can also occur if carrying out two separate surveys. Another conflicting issue is that a large questionnaire, which can be hard to interpret, may result in an increase of the response burden, but large and complicated questionnaires can also have positive side. One could argue that the more extensive and accurate a questionnaire is the questionnaire the more accurate answers you get hence it improves the support given to the respondent e.g. the interpretation of the distinction between the concepts of R&D and innovation. On the other hand it may be likely to confuse due to the numerous examples on concepts that may not seem equally relevant to all of the respondents and thereby add to the experienced response burden. Or these respondents may simply fail to answer thus reducing the liability of the data. Consequently the joint system of terminology could also be considered a weakness in the survey design. It is worth mentioning that the above listed points on advantages and disadvantages, has been made of the basis of a paper questionnaire being the primary media of data collection. Due to the technological advancement with web-based media the importance of some of the disadvantages are diminished. As described above, some issues may be considered both gains and weaknesses when evaluating the Danish R&D&I approach. Some of these issues can be discussed easily and some need further elaboration. 9

Analysis The issues presented in the previous chapter will be addressed in the following text. There are two sources of information used in the analysis: the data from the Danish R&D&I statistics from 2007-2010 and the data from voluntary user survey made as a follow up to the 2010 data collection. Data 2007-2010 (Part 1) By using the data from Danish R&D&I statistics from 2007-2010 we can present an analysis on various distributions in R&D&I. In 2010 21 per cent of the 4.322 7 responding units reported both R&D and innovation activities. 41 per cent reported to have neither R&D nor innovation activities. 35 per cent reported only to have innovation activities and no R&D and 3 per cent reported that their enterprise only engaged in R&D and besides that no innovation activities. Figure 1 Activities per year 100 Per cent 80 60 40 R&D only Innovation only Both R&D and Innovation No R&D and Innovation 20 0 2007 2008 2009 2010 These shares of reported activities show rather small deviations between the years. 2008 differs from the other years as the share of enterprises with both R&D and innovation activities increases and the share of enterprises with innovation only decreases. The shares show that in 2010 21 per cent of the population had both R&D and innovation activities to report. 41 per cent had no R&D or innovation activities and 35 per cent reported to have some sort of innovation, but no R&D. Only 3 per cent of the respondents reported to have only R&D and no innovation. Estimated time use It has been argued that the joint R&D&I survey would imply a reduced response burden. This is tested with a simple calculation using a model with estimated time use per reported element (e.g. intra- or extramural R&D, innovation). As part of the evaluation of the 2007 data collection an estimate on the used time per response was 7 Un-weighted values 10

given. These estimates have been adjusted to fit both even and odd years and the values are presented in table 4. Table 4. Minutes spend reporting the joint R&D&I survey. Minutes spend reporting elements of the joint R&D&I survey Even years Odd years Internal R&D 360 180 External R&D 120 60 Innovation activities 240 240 None of the above 30 30 Following the estimated levels of time used per report the total use of time per data collection can be calculated. In Figure 2 the time use per data collection is presented with time spend filling out measured in working weeks. The figure shows that the data collection in 2009 was most costly in terms using the most resources measured in working weeks. For all the years the main contributor to the total use of time was enterprises reporting to have innovation activities. In 2009 their share of the total time spend was 53 per cent and the value in 2010 was 68 per cent. When comparing even and odd years, the main reason to the differences between the shares of time spend is the R&D accounting for 36 per cent in 2009 dropping to 22 per cent in 2010. Figure 2. Estimation of total time (weeks of work 8 ) spent reporting activities using the combined surveys approach. Based on existing data and combined surveys. Minutes on reporting (CS) 500 400 No R&D and innovation 300 Innovation reporting 200 External R&D reporting Internal R&D reporting 100 0 2007 2008 2009 2010 The calculation of the total time used on reporting is repeated, but in a scenario with individual surveys will be as follows. The estimated time use per report with two individual surveys is as shown in table 5. 8 A full working week is here defined as 37 hours. 11

Table 5.. Minutes spend reporting the separated surveys. Even years Odd years R&D CIS R&D CIS Internal R&D 180 360 External R&D 60 120 Innovation activities - 240-240 None of the above 20 25 20 25 The only element that has been changed is the reporting of no R&D and no innovation. This is because that it is assumed that any reporting takes up some time for preparation, e.g. opening the envelope with the paper questionnaire or log on to the web-based media, before ticking no to all filter questions. With these estimated values of time used per report the total time spend per data collection is as follows in figure 3. Figure 3 Estimation of total working weeks spend reporting activities with individual surveys. Based on model with individual surveys. 600 Minutes on reporting (IS) 500 No R&D and innovation 400 Innovation reporting 300 External R&D reporting 200 Internal R&D reporting 100 0 2007 2008 2009 2010 Comparison of the results in figure 2 and figure 3 shows that the cost of separating the R&D&I survey to two individual surveys will be an increase in the burden measured in time spent per enterprise reporting data. Combination of R&D and innovative enterprises An advantage with the joint R&D&I surveys is that it makes it possible to analyse the way enterprises use R&D to innovate. Figure 4 presents share of innovative enterprises with their level of R&D activities. In 2010 84 per cent of the innovative 12

enterprises had intramural R&D work. The share of innovation within enterprises with no R&D was just 36 per cent. This is below the average for all enterprises that had a share of innovation at 47 per cent. Figure 4 Share of enterprises with innovation, distributed by type of R&D-activity. 2010 100 Per cent. 80 84 87 90 89 87 60 40 20 36 47 0 In-house R&D Permanent R&D staff External R&D Acquisition of patents, knowhow Apllied for patents Enterprises with no R&D activities All enterprises 13

Review from respondents 2010 (Part 2) As a follow up on the 2010 data collection respondents using the web-based questionnaire were asked to report data about the experience. This user survey was collected on a voluntary basis. The review questions were: How difficult did you find it to understand the questions and help texts? (scale 1-4 9 ) How difficult did you find it to gather the necessary information? (Scale1-4) How much time did you spend reporting? (hours/minutes) The above mentioned review questions are in the following text combined with the reported activity level for the R&D and Innovation Time spent 2.317 enterprises responded to the review questions on time spent. These data are combined with the answers from the R&D&I survey (2010). In figure 5 the average values per combination of R&D&I activities are presented. Not surprisingly the lowest average value is 24 minutes for enterprises with no R&D and innovation and the highest value is 198 for enterprises with both intramural, extramural and innovation. Enterprises with Intramural and extramural R&D and no innovation have the second highest value with 159 minutes. The four highest values all contain Intramural R&D suggesting that reporting the intramural R&D make up a big part of the total time spend on reporting R&D&I statistics. Figure 5 Average time spent reporting R&D&I activities. 2010 250 200 Minutes No R&D or innvation Innovation only Intramural R&D only Intra- and extramural (no innovation) Extramural R&D only Extramural R&D and innovation (no intramural R&D) Intramural R&D and innovation (no extramural) Intra-, extramural and innovation 198 150 159 100 90 107 114 50 49 67 0 24 The distribution of reported time use presented in figure 5 is somewhat distant from the values used calculating the estimated time use as presented in figures 2 and 3 suggesting a revision of the estimated values. 9 The four points on the scale is : 1) Very difficult 2) difficult 3) a little difficult 4) not difficult at all 14

Figure 6 Total time spent reporting R&D&I ativities. 2010 65 60 55 Minutes 60.564 No R&D or innvation Extramural R&D only 50 45 Innovation only 40 35 Extramural R&D and innovation (no intramural R&D) 30 25 20 15 10 5 0 20.626 340 31.147 32.495 2.970 5.332 3.825 Intramural R&D only Intramural R&D and innovation (no extramural) Intra- and extramural (no innovation) Intra-, extramural and innovation In total time spent (in minutes) per combination of R&D&I the biggest contributor is enterprises with innovation with 61,000 minutes followed by enterprises with both intra-, extramural and innovation and extramural R&D and innovation (no intramural R6D) who adds up for respectively 32,000 and 31,000 minutes. If the total time spent is compared to the number of reporting enterprises, see figure 7, it is obvious that enterprises with no types of R&D and innovation makes up for a large share of the number of enterprises, but only a small share of the time spent. The inverse situation is seen in enterprises with intramural R&D and innovation (and no extramural R&D). Figure 7 Share of spent time and respondents reporting R&D&I activities. 2010 100 90 80 70 60 50 40 Per cent Intramural R&D and innovation (no extramural) Extramural R&D and innovation (no intramural R&D) Intra-, extramural and innovation Intra- and extramural (no innovation) Innovation only 30 Extramural R&D only 20 10 Intramural R&D only 0 Share of minutes Share of respondents No R&D or innvation 15

Understanding the questions 2,419 responded to the How difficult was it to understand the questions and help texts- question. The distribution of the difficulties experienced regarding the understanding of the substance of the questionnaire in connection to the various types of R&D&I activities is presented in Figure 8. Figure 8 Difficulties with understanding questions and help texts reporting R&D&I. On R&D&I activities 10. 2010 100 Per cent Very difficult Difficult A little difficult Not difficult 80 60 40 20 0 Extramural R&D and innovation (no intramural R&D) Intra- and extramural (no innovation) Intra-, extramural and innovation Extramural R&D only Intramural R&D and innovation (no extramural) Intramural R&D only Innovation only Average No R&D or innvation The average rating of all respondents was that 24 per cent found it either very difficult or difficult to understand the questions presented in the survey leaving 76 per cent with the perception that it was only a little difficult or not difficult at all to understand the questions. The group of respondents who experienced the least difficulties was the enterprises with no types of R&D or innovation, where 82 per cent of the respondents reported to have no or little difficulties. Gather the information needed The How difficult did you find it to gather the necessary information? was reported by 2417 enterprises. The distribution of the difficulties in connection to find the information needed to complete the questionnaire for the various types of R&D&I activities is presented in Figure 9. 10 The category extramural R&D only should be disregarded since this category only consists of 6 records 16

Figure 9 Difficulties for gathering the needed information for reporting R&D&I activities 11. 2010 100 Per cent Very difficult Difficult A little difficult Not difficult 80 60 40 20 0 Intra-, extramural and innovation Extramural R&D and innovation (no intramural R&D) Extramural R&D only Intra- and extramural (no innovation) Intramural R&D and innovation (no extramural) Intramural R&D only Innovation only Average No R&D or innvation Not surprisingly, the enterprises reporting most elements to the R&D&I also experience most difficulties with gathering the information needed to report the demanded data. 65 per cent of enterprises with both intramural, extramural and innovation activities to report experienced it to be very difficult or difficult to gather the necessary information. Enterprises with innovation only have a share of 35 per cent finding it difficult or very difficult to gather the information. The share for enterprises with intramural R&D only is at 37 per cent and thus almost the same level as the former category. 11 The category Extramural R&D only should be disregarded since this category only consists of 5 records 17

Conclusions Many of the identified challenges with an extensive questionnaire diminish with the advances of the dynamic web-based questionnaire. Issues such as the size of the questionnaire are diminished since the web-based questionnaire offers solutions where only the relevant filter questions are shown at first sight. Enterprises with no R&D & innovation activities only need to tick no to these question and thus do not experience the entire selection of modules and questions. The vast majority of the respondents only find it a little difficult or not difficult at all to understand the questions in the questionnaire. The respondents with extensive reporting in all aspects of the questionnaire s modules mainly experience it to be difficult to gather the information to fill in the questionnaire. On basis of this knowledge it should be investigated if new ways to support the enterprises experiencing difficulties gathering the needed information could be launched. The joint Danish R&D&I survey implies economies of scale in relation to the data collection and processing. Fewer request to report and reminders are sent to the respondents. When potential errors are detected in the questionnaires the follow up contact to the enterprise requires fewer resources as one call can clarify the issues. The joint data collection and processing gives a good foundation for the work with validation of the data. In relation to detecting errors in the data material the joint data offers information that helps identify various errors in the reported from the enterprises due to misinterpretations or other types of errors. The comparison of the estimated time used show that a split up from the joint survey to two individual surveys fitting the survey would increase the response burden. And one should have in mind that a large share of the responding units have both R&D and innovation. Road ahead Statistics Denmark has identified potential areas of improvement and in the future these areas of interest will be further analysed. On the basis of the advantages of the Danish joint R&D&I Statistics Denmark will continue to use this approach and will continue the work with improvements of the survey design. The dynamic web-based questionnaire has a great potential in relation to improving the conditions for the respondents and hence the quality of data. In the future joint R&D&I surveys Statistics Denmark will continue to work on the improvement of the web-based questionnaire e.g. improving the routing and support. The majority of the responding enterprises do not find it particularly difficult to understand the questions of the survey. Enterprises with R&D and innovation to report do have difficulties especially regarding the gathering of the needed information. Statistics Denmark will investigate if there are potential improvement gains e.g. by making the support- and help text more specific in relation to the given industry. The user reviews offers valuable knowledge and these data will continuously be collected and used as input to the process of improving the survey design. 18

Number of responses by filter questions. Statistics on R&D and Innovation, Denmark 2009 2) Own R&D? 4,545 no 3,466 yes 1,079 14) Product innovation? 4,545 no 3,283 yes 1,262 15) Process innovation? 4,545 no 3,310 yes 1,235 16) Ongoing or abandoned innovation projects? 4,545 no 3,540 yes 1,005 17) Organisational innovation? 4,545 no 2,894 yes 1,651 18) Marketing innovation? 4,545 no 3,173 yes 1,372 Annex 1 - Overview of routing in the 2009 questionnaire 14a) main developer 15a) main developer 17a) main objectives 18a) main objectives 14b) degree of novelty 1,235 1,651 1,372 14c) turnover by degree of novelty 1,262 2) types of R&D 2a)+2b) No. of R&D f.t.e.+r&d personel 3) R&D in foreign affiliates 4) Expenditure on intramural R&D 5)+6) R&D expenditure by type of R&D 7) Sources of financing of intramural R&D 8) Expenditures for intramural R&D by R&D area 9) Expenditures on R&D by end product 10) Expenditures on extramural R&D 11) Sources of financing of extramural R&D 12) R&D co-operation 13) R&D within primary type of activity? 1,079 16a) main objectives for innovation activities 1,903 19)+19a) Cooperation on innovation activities? 20)+21) Sources of information for innovation activities? 22) Importance of own R&D, data on customers/users needs, cooperation on innovation etc. for innovationa activities? 23) Education or training as part of innovation activities? 24) Geographical location of innovation activities? 25) Innovation expenditures? 2,478 Note: Numbers in bracket - e.g. 14) - refer to number in questionnaire. Numbers in blue refer to the number of responses. Arrows indicate the routing, e.g.: no matter if the response to product innovation is 'yes' og 'no', respondents have to fill in the question on process innovation. But if yes to product innovation, respondents have to fill in also questions on type of product innovation etc.

Annex 2 - The Danish 2009 questionnaire + Forskning, udvikling og innovation 2009 Foretag indberetningen online på Virk.dk Når du indberetter på Virk.dk, får du hjælp undervejs. Din indberetning kan printes, gemmes, genåbnes og rettes. Omfattede CVR-numre Besvarelsen skal kun omfatte den udvalgte juridiske enhed. Der kan dog svares for flere CVR-numre: hvis virksomhedens forsknings- og innovationsaktiviteter omfatter flere juridiske enheder hvis der svares på koncernniveau I begge tilfælde skal der kun svares for danske virksomheder. 1. Omfatter besvarelsen flere CVR-numre? Nej, kun det CVR-nummer, som står på skemaet Ja, flere CVR-numre a. Hvilke CVR-numre er omfattet af indberetningen? CVR-nummer Navn Hvis flere end 4 CVR-numre, vedlæg da en liste. Undersøgelsens indhold 1244101-01 C Undersøgelsen består af to emner: - Forskning og udvikling, der behandles i spørgsmål 2-13. - Innovation, der behandles i spørgsmål 14-25. Forskning og udvikling (FoU) er skabende arbejde foretaget på et systematisk grundlag for at øge den eksisterende viden, og udnyttelsen af denne viden til at udtænke nye anvendelses-områder. Fælles for FoUaktivitet er, at den skal indeholde et væsentligt nyhedselement. Innovation er selve introduktionen eller implementeringen af nye eller væsentligt forbedrede produkter, produktionsprocesser eller markedsføringsmetoder, samt væsentlige organisatoriske ændringer. I 2008 havde 42 af virksomhederne mindst én form for innovation. Der er mange flere eksempler at finde i vejledningen til undersøgelsen på www.dst.dk/forskninginnovation Sejrøgade 11 Dataindsamling e-post indberetning@dst.dk + + 2100 København Ø Direkte tlf. 39 17 32 80

+ + Virksomhedens forskning og udviklingsarbejde (FoU) FoU er: Skabende arbejde foretaget på et systematisk grundlag for at øge den eksisterende viden, og udnyttelsen af denne viden til at udtænke nye anvendelsesområder. FoU kan inddeles i Grundforskning Anvendt forskning Udviklingsarbejde FoU kan fx angå... it, teknik og fremstillingsvirksomhed jordbrug, fødevarer og veterinærområdet administration og økonomisk planlægning salg og marketing. 2. Har virksomheden i 2009... - købt FoU-arbejde fra andre? herunder fra andre dele af koncernen... Ja Nej Sæt ét kryds i hver linje - købt eller indlicenseret patenter, brugsmodeller, varemærker, designs eller knowhow?... - selv søgt patenter?... Hvis Nej til begge: Gå til spm. 14, side 11 - haft egen forsknings- eller udviklingsafdeling?... - selv udført eget FoU-arbejde i Danmark?... Hvis Ja til købt FoU og Nej til eget FoU-arbejde: Gå til spm. 10, side 8 Hvilke former for FoU har virksomheden udført i 2009? Ja Nej - grundforskning? dvs. eksperimenterende eller teoretisk arbejde med det primære formål at opnå ny viden og forståelse uden nogen bestemt anvendelse i sigte... - anvendt forskning? dvs. eksperimenterende eller teoretisk arbejde med det formål at opnå ny viden og forståelse. Arbejdet er dog primært rettet mod bestemte anvendelsesområder... - udviklingsarbejde? dvs. systematisk arbejde baseret på anvendelse af viden opnået gennem forskning og/eller praktisk erfaring med det formål, at frembringe nye eller væsentligt forbedrede materialer, produkter, processer, systemer eller tjenesteydelser... 2 + +

+ + FoU-personale og årsværk FoU-personale Ansatte der har arbejdet med forsknings- og udviklingsopgaver i mindst 5 af deres arbejdstid dvs. mindst 80 timer pr. år for en fuldtidsansat FoU-årsværk: Én Én persons arbejde med FoU på fuld tid i et år udgør 1 årsværk. persons arbejde med FoU på halv tid i et år udgør 0,5 årsværk. 2a. Hvor meget af virksomhedens personale har udført, støttet eller administreret FoU i Danmark i 2009? Forskere og andre specialister der har udviklet ny viden, produkter, processer, metoder eller systemer eller planlagt og ledet FoU-projekter... Personer i 2009 Heraf kvinder FoUårsværk i 2009, Heraf kvinder, Teknisk personale der har støttet FoU-projekter med teknisk viden og kompetence. Fx maskinmestre, laboranter, programmører mm....,, Andet personale der har serviceret FoU-projekter. Fx håndværkere, kontorpersonale, maskinoperatører mm....,, I alt beskæftiget med FoU i 2009...,, 2b. Er der nogen ansatte forskere med FoU-opgaver der har en ph.d.-, licentiat- eller doktorgrad, er i gang med en erhvervs-ph.d. eller er udenlandske forskere? Ja Nej Gå til spm. 3 Angiv antal ansatte og beregnet antal FoU-årsværk Personer i 2009 Heraf kvinder FoUårsværk i 2009 Heraf kvinder Forskere med ph.d.-, licenciat- eller doktorgrad,, Erhvervs-ph.d.'ere,, Forskere med udenlandsk statsborgerskab,, 3. Har virksomheden i 2009 haft FoU-aktiviteter i koncernens udenlandske datterselskaber? Ja Nej Gå til spm. 4 Antal ansatte med FoU-opgaver : Skønnet antal ansatte i udenlandske datterselskaber der udfører FoU-aktiviteter for virksomheden fordelt på hjemsteder... Europa USA Kina/ Indien Udland i øvrigt + 1244101-02 C 3 +

+ + Udgifter til egen FoU i Danmark i 2009 Vejledning Opgør poster så de samlede udgifter til FoU giver et retvisende billede. Skriv 0 ud for de enkelte poster, hvis virksomheden ikke har haft nogen udgifter. 4a. Driftsudgifter FoU-aktiviteter: - Løn og sociale udgifter vedr. FoU Inkl. ATP og feriepenge... - Andre driftsudgifter vedr. FoU Fx husleje, lys, varme, rengøring, rejser, kontorhold, mindre apparatur, materialer og it-kørsler... Ekskl. moms i hele 1 kr 4b. Anlægsudgifter FoU-aktiviteter: - Bygninger anvendt til FoU-arbejde Fx nybygninger, større ombygninger, køb af ejendom. Faktiske omkostninger uden afskrivninger... - Apparatur og instrumenter til FoU Med købspris over 50 og flerårig levetid... 4c. Udgifter til egen FoU i alt... Fordeling af driftsudgifter til egen FoU 5. Angiv hvordan driftsudgifterne til egen FoU er fordelt på... Andel i - Grundforskning - Anvendt forskning - Udviklingsarbejde I alt 100 6. Angiv hvordan driftsudgifterne til egen FoU er fordelt på... Andel i - Produktorienteret forskning og udviklingsarbejde - Procesorienteret forskning og udviklingsarbejde - Almen vidensudvikling mv. I alt 100 4 + +

+ + Finansiering af samlede udgifter til egen FoU i 2009 7. Fordel udgifter til egen FoU i 2009 på finansieringskilder: a. Intern finansiering Ekskl. moms i hele 1 kr - Virksomhedens egen finansiering af egen FoU... b. Andre danske finansieringskilder - Danske virksomheder i samme koncern... - Andre danske virksomheder - inkl. ventureselskaber... - Private danske organisationer og fonde mv.... - Videnskabsministeriet (Højteknologifonden, Rådet for Teknologi og Innovation, Det Frie Forskningsråd, Innovationsmiljøer)... - Andre statslige institutioner... - Regioner og kommuner... - Vækstfonden... c. Udenlandske finansieringskilder - Udenlandske virksomheder i samme koncern... - Andre udenlandskevirksomheder... - Private udenlandske organisationer og fonde mv.... - EU-midler - Anden offentlig udenlandsk finansiering... - Fordelt finansiering i alt (Skal svare til de samlede udgifter til FoU som er opgjort i spm. 4c)... 1244101-03 C 5 + +

+ + Udgifter til egen FoU fordelt på forskningsområder Medtag evt. udgifter til samme FoU-aktivitet under flere områder Summen af de fordelte udgifter kan være større eller mindre end 100 8. Fordel de samlede udgifter til egen FoU på udvalgte forskningsområder: Andel i Materialeforskning... Bygge- og anlægsteknik... Sundhedsforskning... Genteknologi... Bioteknologi... Nanoteknologi... Levnedsmiddelforskning... Energiforskning... Miljøforskning... Ældreforskning og hjælpemiddelforskning... Forsvarsteknologi... Ledelses-, organisations-, kompetenceforskning... Programmel integreret i andre produkter... Programmel som selvstændige produkter... Hardware... Andre områder... 6 + +

+ + Udgifter til egen FoU fordelt på produktgrupper 9. Fordel de samlede FoU-udgifter på de produkter eller tjenesteydelser som virksomhedens FoU har været rettet imod Skøn andel af samlet FoU-udgift som er anvendt til... Råvarer fra landbrug, skovbrug og fiskeri... Andel i Industrielle processtyringsanlæg, navigationsudstyr, måle- og kontrolapparater... Andel i Råstoffer og udvinding: olie, gas, kalk mv.... Ure, medicinsk, kirurgisk, optisk og fotografisk udstyr... Nærings- og nydelsesmidler... Transportmidler, inkl. biler, skibe og fly Tekstiler, beklædning og lædervarer... Møbler... Trævarer, papir, pap og grafiske produkter... Andre produkter indenfor fremstilling. Raffinerede olie- og kulprodukter... El-, gas-, varme- og vandforsyning... Medicinalvarer... Bygge- og anlægsvirksomhed... Andre kemiske produkter... Gummi- og plastprodukter... Sten-, ler- og glasprodukter... Jern, stål og andre metaller... Varer af jern- og metal... Motorer og motordele - ekskl. motorer til flyvemaskiner, motorkøretøjer, knallerter Landbrugs- og skovbrugsmaskiner... Maskiner til generelle formål samt industri- og værktøjsmaskiner... Våben og ammunition... Husholdningsapparater... Kontormaskiner og it-udstyr... Isolerede ledninger og kabler... Elektriske motorer, apparater og materiel i øvrigt... Radio, tv, højttaler, antenner mv.... Elektroniske komponenter og telemateriel i øvrigt... Engroshandel med radio, tv, kontorog it-udstyr, el-installationsmateriel og elektroniske komponenter... Reparation og vedligeholdelse af kontormaskiner og it-udstyr... Handel og reparationsydelser af kontormaskiner og it-udstyr... Hotel og restaurationsydelser... Transporttjenester... Post- og kurértjenester... Telekommunikation... Udlejning af it-udstyr og -maskiner, samt kontormøbler... Tjenester vedr. finansiering, forsikring, fast ejendom og udlejning i øvrigt... Databehandling inkl. konsulentvirksomhed og udvikling af hardware.. Udvikling af software... Andre serviceydelser... Andet... I alt... 100 1244101-04 C 7 + +

+ + Udgifter til købt FoU udført af eksterne aktører i 2009 Vejledning Opgør poster så de samlede udgifter giver et retvisende billede. Skriv 0 ud for de enkelte poster, hvis virksomheden ikke har haft nogen udgifter. 10. Udgifter til købt FoU FoU købt af virksomheder... - i samme koncern i Danmark FoU udført af andre dele af koncernen i Danmark og anvendt internt i virksomheden... - i samme koncern i udlandet FoU udført af andre dele af koncernen i udlandet og anvendt internt i virksomheden... Ekskl. moms i hele 1 kr FoU købt i Danmark fra... - andre virksomheder... - konsulenter... - godkendte teknologiske serviceinstitutter (GTS) Se liste: www.teknologiportalen.dk/omgts/... - universiteter og andre højere læreanstalter... - andre offentlige forskningsinstitutioner... - Andre, anfør hvilke: FoU købt i udlandet fra... - andre virksomheder... - konsulenter... - offentlige forskningsinstitutioner, fx universiteter... Udgifter til købt FoU i alt... 8 + +

+ + Udgifter til købt FoU fordelt på finansieringskilder 11. Fordel udgifter til købt FoU i 2009 på finansieringskilder: a. Intern finansiering Ekskl. moms i hele 1 kr - Virksomhedens egen finansiering af købt FoU... b. Andre danske finansieringskilder - Danske virksomheder i samme koncern... - Andre danske virksomheder - inkl. ventureselskaber... - Private danske organisationer og fonde mv.... - Videnskabsministeriet (Højteknologifonden, Rådet for Teknologi og Innovation, Det Frie Forskningsråd, Innovationamiljøer)... - Andre statslige institutioner... - Regioner og kommuner... - Vækstfonden... c. Udenlandske finansieringskilder - Udenlandske virksomheder i samme koncern... - Andre udenlandskevirksomheder... - Private udenlandske organisationer og fonde mv.... - EU-midler - Anden offentlig udenlandsk finansiering... - Fordelt finansiering i alt (Skal svare til de samlede udgifter til købt FoU som opgjort i spm. 10).. 1244101-05 C 9 + +

+ + Samarbejde om forsknings- og udviklingsprojekter Aktivt samarbejde med andre virksomheder og institutioner om FoU-projekter. Medtag FoU-samarbejde uanset om det giver umiddelbare kommercielle fordele. Medtag ikke ren udlicitering af FoU-aktiviteter uden aktivt samarbejde. 12. Har virksomheden i 2009 samarbejdet med eksterne aktører om virksomhedens FoU-arbejde? Ja Nej Gå til spm. 13 a. Angiv samarbejdspartnere og deres hjemsteder Sæt mindst ét kryds for hver type samarbejdspartner Danmark Europa i øvrigt USA Kina/Indien Udland i øvrigt Ikke relevant Andre virksomheder i koncernen... Leverandører af udstyr, materialer, dele eller software.. Kunder eller klienter... Konkurrenter eller andre virksomheder i samme branche GTS (Godkendte Teknologiske Serviceinstitutter)... Private FoU-virksomheder, fx konsulenter, private laboratorier og - forskningsinstitutioner... Virksomheder i andre brancher ekskl. kunder og leverandører... Universiteter eller andre højere læreanstalter... Offentlige forskningsinstitutioner... Offentlige serviceudbydere... fx sygehuse, skoler, daginstitutioner, forsvaret mv. Andre offentlige samarbejdspartnere... Hovedbranche for FoU-aktiviteter Nej Ja 13. Foregik virksomhedens FoU-aktiviteter i 2009 primært inden for virksomhedens hovedbranche? Gå til spm. 14 a. Angiv den branche som virksomhedens FoU-aktiviteter primært udføres i: Virksomheden er registreret i branche: 10 + +