2005-evaluering Slutevaluering af Mål 3-programmet Februar Erhvervs- og Byggestyrelsen og De Europæiske Fællesskaber DEN EUROPÆISKE UNION

Transkript

1 Erhvervs- og Byggestyrelsen og De Europæiske Fællesskaber 2005-evaluering Slutevaluering af Mål 3-programmet Februar 2006 DEN EUROPÆISKE UNION Den Europæiske Socialfond

2 Slutevaluering af Mål 3-programmet Side i 1. INTRODUKTION Evalueringens genstandsfelt Evalueringens hovedspørgsmål og læsevejledning Bilagsmateriale Forkortelser EXECUTIVE SUMMARY Programme contents, context and implementation Programme outcome and impact Long-term effects of the total Objective 3 programme Special focus on long-term effects of priorities 3 and Prerequisites for increased outcome and impact Added value in relation to EES Follow-up on the mid-term evaluation and recommendations for the future SAMMENFATNING Programmets indhold, kontekst og implementering Programmets outcome og impact Langsigtede effekter af det samlede Mål 3-program Særligt fokus på langsigtede effekter af prioritet 3 og Forudsætninger for øget outcome og impact Merværdi i forhold til EES Opfølgning på midtvejsevalueringen og fremadrettede anbefalinger PROGRAM OG KONTEKST Mål 3-programmets indhold, målgrupper og finansielle vægt Indhold, prioriteter og målgrupper Finansiel vægt Mål 3-programmets politiske og socioøkonomiske ramme Mål 3-programmets europæiske politiske kontekst Mål 3-programmets nationale politiske kontekst Mål 3-programmets socioøkonomiske kontekst Mål 3-programmets understøttelse af relevante politikker Mål 3-programmet og Lissabon-strategien/EES Mål 3-programmet og NAP/NAPincl... 40

3 Slutevaluering af Mål 3-programmet Side ii Mål 3-programmet og dansk arbejdsmarkedspolitik Mål 3-programmet og Vækst med vilje OPFØLGNING OG FREMADRETTEDE ANBEFALINGER Opfølgning på midtvejsevalueringens anbefalinger Hovedudfordring 1: Erfaringsopsamling og videnspredning Hovedudfordring 2: Samordning med øvrige nationale strategier Hovedudfordring 3: Administrative rutiner på tværs Hovedudfordring 4: Videreudvikling af indikatorer Hovedudfordring 5: Styrkelse af POU s arbejde Anbefalinger til den kommende ESF-periode PROGRAMMETS IMPLEMENTERING Programmets finansielle gennemførelse Finansiel gennemførelse på prioritets- og foranstaltningsniveau Finansiel gennemførelse for hhv. individ- og systemrettede foranstaltninger Iværksatte aktiviteter under Mål 3-programmet Iværksatte aktiviteter under Mål 3-programmets individrettede foranstaltninger Iværksatte aktiviteter under Mål 3-programmets systemrettede foranstaltninger Mål 3-programmets output Individrettede foranstaltningers output Systemrettede foranstaltningers output Den iværksatte informationsstrategi Implementeringsmekanismer Programovervågningsudvalget Programadministrationen Projektadministrationen Revision og controlling OPUS Samlet evaluering af Mål 3-programmets implementering, output og opfølgning på informationer fra monitorerings- og evalueringssystemet Iværksatte aktiviteter og finansiel gennemførelse Implementeringsmekanismer og opfølgning på informationer fra monitorerings- og evalueringssystemet PROGRAMMETS RESULTATER Individrettede foranstaltningers outcome og impact Individrettede foranstaltningers indhold Indikatorer for individrettede foranstaltningers outcome Individrettede foranstaltningers outcome... 84

4 Slutevaluering af Mål 3-programmet Side iii Indikatorer for individrettede foranstaltningers impact Individrettede foranstaltningers impact Særligt fokus på foranstaltning 2.1.A s og 2.1.B s impact Foranstaltning 2.1.A s og 2.1.B s beskæftigelseseffekter Foranstaltning 2.1.A s og 2.1.B s øvrige former for impact Samlet vurdering af foranstaltning 2.1.A og 2.1.Bs impact Systemrettede foranstaltningers outcome og impact Systemrettede foranstaltningers indhold Indikatorer for systemrettede foranstaltningers outcome og impact Systemrettede foranstaltningers outcome og impact Samlet vurdering af Mål 3-programmets outcome og impact FOCUSED APPROACH Valget af prioritet 3 og 4 til dybdegående analyse Individrettede foranstaltninger Foranstaltning 3.2: Fremme af ansattes kvalifikationer og fleksibilitet Foranstaltning 4.1.A: Udvikling af en iværksætter- og innovationskultur, kompetencer Foranstaltning 4.1.B: Udvikling af en iværksætter- og innovationskultur, netværk Systemrettede foranstaltninger Foranstaltning 3.1.A: Foregribelse af arbejdsmarkedsudviklingen og effektvurdering Foranstaltning 3.1.B: Effektvurdering og analyser af beskæftigelsespolitiske tiltag Foranstaltning 3.3: Tilpasning, fleksibilitet og livslang læring Foranstaltning 4.1.C: Udvikling af en iværksætter- og innovationskultur, systemer Implementeringssystemets indflydelse på resultaterne af prioritet 3 og PROGRAMMETS MERVÆRDI I RELATION TIL IMPLEMENTERINGEN AF EES Mål 3-programmets merværdi i forhold til EES på europæisk plan Mål 3-programmets merværdi i forhold til EES på nationalt plan Merværdi i forhold til NAP og NAPincl Merværdi i forhold til Flere i arbejde med særligt fokus på prioritet 1 og Merværdi i forhold til Vækst med vilje med særligt fokus på prioritet 3 og Mål 3-programmets merværdi i forhold til EES på lokalt plan

5 Slutevaluering af Mål 3-programmet Side iv 9.4 Mål 3-programmets merværdi i forhold til EES på individuelt plan MAINSTREAMING AF PROGRAMMET Horisontal og vertikal mainstreaming af programmets resultater Mainstreaming af kønsligestilling under Mål 3-programmet Forudsætningerne for mainstreaming af kønsligestilling Den faktiske mainstreaming af kønsligestilling Mainstreamingens effekter på kønsligestilling

6 Slutevaluering af Mål 3-programmet Side 1 1. INTRODUKTION 1.1 Evalueringens genstandsfelt Mål 3-programmet udgør ét af Socialfondens tre programmer og er et vigtigt finansielt redskab til udvikling af de menneskelige ressourcer i forbindelse med den fælles beskæftigelsesstrategi, EES. Formålet med det danske Mål 3-program er at bekæmpe og forebygge ledighed, højne kvalifikationsniveauet blandt ansatte samt styrke udviklingen af en iværksætter- og innovationskultur, jf. de fire prioriteter. Programmet arbejder ud fra en række fælleseuropæiske kriterier, men tilpasses samtidig de særlige nationale behov og arbejdsmarkedsstrukturer. Danmark modtager ca. 419 millioner kr. om året fra Socialfonden, hvoraf ca. 407 mio. kr. er til projekter, mens ca. 12 mio. kr. er øremærket teknisk assistance (TA-midler) 1. Midlerne administreres centralt af Erhvervs- og Byggestyrelsen (EBST) og regionalt af amterne, Bornholms Regionskommune samt Københavns- og Frederiksberg kommuner. Denne rapport er et kondensat af slutevalueringen af Mål 3-programmet i Danmark, som er gennemført i perioden fra januar 2005 til januar Der er tale om en programevaluering, hvor det er program- og projektadministrationen samt programmets samlede effekt, som evalueres. Hvor det i midtvejsevalueringen primært var muligt at vurdere programmets effekter på kort sigt, søges det i slutevalueringen at vurdere programmets mere langsigtede effekter. For en uddybning af rapportens konklusioner og det datagrundlag, disse er baseret på, henvises løbende til en række bilag (for en oversigt, se afsnit 1.3). Evalueringen er udarbejdet af NIRAS Konsulenterne og DMA/Research i samarbejde med revisionsfirmaet KPMG på vegne af Erhvervs- og Byggestyrelsen (EBST). 1.2 Evalueringens hovedspørgsmål og læsevejledning I henhold til den danske kravspecifikation, COM Guidance Paper on Final Evaluation, samt aftale mellem EBST og evaluator om særlige danske fokusområder, skal slutevalueringen af Mål 3-programmet i Danmark berøre 11 forskellige punkter 2. I nærværende kondensat af slutevalueringen er kravspecifikationens 11 punkter systematiseret efter COM In- 1 Tallene er beregnet på baggrund af data tilsendt fra EBST d. 4/ De samlede ESF-midler under Mål 3- programmet i Danmark i perioden udgør således EUR Arbejdsmarkedsstyrelsen (2002): Kravspecifikation for evaluering af Den Europæiske Socialfonds Mål 3- program i perioden , København: Arbejdsmarkedsstyrelsen

7 Slutevaluering af Mål 3-programmet Side 2 formation extraction grid for the synthesis of final evaluations (se bilag 1 for en oversigt over, i hvilke kapitler og øvrige bilag evalueringens hovedspørgsmål er søgt besvaret). Rapporten har således følgende opbygning: Kapitel 2 og 3 indeholder en kort dansk og engelsk sammenfatning af evalueringens konklusioner med fokus på programmets resultater og merværdi, samt anbefalinger for den kommende programperiode. Kapitel 4 behandler Mål 3-programmets indhold og dets strategiske samspil med den politiske og socioøkonomiske kontekst, hvori det udfoldes 3. Kapitel 5 indeholder dels en oversigt over midtvejsevalueringens anbefalinger og opfølgningen herpå, dels en oversigt over de anbefalinger, evaluator fremsætter til den kommende Socialfondsperiode I kapitel 6 evalueres Mål 3-programmets implementering. Kapitlet indeholder således en vurdering af programmets finansielle gennemførelse, iværksatte aktiviteter og output samt implementeringsmekanismer 5. Kapitel 7 indeholder en evaluering af Mål 3-programmets outcome og impact med henblik på at vurdere Mål 3-programmets målopfyldelse på mellemlangt og langt sigt. Vurderingen foretages for programmet som helhed, for de individ- og systemrettede foranstaltninger hver især, og særskilt for foranstaltning Kapitel 8 fokuserer også på resultater og effekter, men indeholder en mere dybdegående analyse af foranstaltningerne under Mål 3-programmets prioritet 3 og 4 7. Disse foranstaltninger er udvalgt til dybdegående analyse pga. deres særlige aktuelle og fremadrettede relevans. I kapitel 9 foretages en samlet vurdering af Mål 3-programmets merværdi i forhold til den politiske kontekst, hvori det udspilles herunder med særligt fokus på merværdien i forhold til implementeringen af EES på hhv. europæisk, nationalt, lokalt og individuelt niveau 8. 3 Kapitel 4 svarer dermed indholdsmæssigt til kravene under COM: Information extraction grid for the synthesis of final evaluation, A: Context. 4 Kapitel 5 berører således indholdsmæssigt dette specifikke krav i COM: Information extraction grid for the synthesis of final evaluation, A: Context. 5 Kapitel 6 svarer dermed indholdsmæssigt til COM: Information extraction grid for the synthesis of final evaluation, B: Delivery. 6 Kapitel 7 svarer dermed til COM: Information extraction grid for the synthesis of final evaluation, D: Results and impacts. 7 Kapitel 8 er således en udbygning af COM: Information extraction grid for the synthesis of final evaluation, D: Results and impacts, svarende til EC COM: Guidance Paper on Final Evaluation, herunder specific questions for in-depth measure level analysis of impact and added value. 8 Kapitel 9 berører dermed spørgsmålene vedr. Mål 3-programmets relevans, samspil og merværdi, som de er nævnt under COM: Information extraction grid for the synthesis of final evaluation, A: Context samt D: Results and impacts.

8 Slutevaluering af Mål 3-programmet Side 3 Endelig sætter kapitel 10 fokus på mainstreamingen af Mål 3-programmet 9. På EU-plan er særligt mainstreaming af kønsligestilling et højt prioriteret emne, hvorfor dette behandles særskilt. Eftersom kønsligestilling ikke har spillet samme centrale rolle i det danske Mål 3- program som i øvrige lande, suppleres ovenstående spørgsmål imidlertid med et mere generelt fokus på mainstreaming af programmets resultater. 1.3 Bilagsmateriale Til uddybning af rapportens analyser og konklusioner findes følgende indholdsmæssige bilag: Bilag 1 Bilag 2 Bilag 3 Bilag 4 Bilag 5 Bilag 6 Bilag 7 Bilag 8 Evalueringens hovedspørgsmål Metodisk fokus og datagrundlag Politisk og socioøkonomisk kontekst Finansiel implementering, output, outcome og impact Implementeringsmekanismer, program- og projektadministration Socioøkonomisk rammesætning Det gode projekt Litteratur Endvidere er rapportens kvantitative datagrundlag dokumenteret i følgende tekniske bilag: Bilag A Bilag B Bilag C Bilag D Spørgeskemaundersøgelse blandt samtlige Mål 3-projekter Projektdeltagernes besvarelse af ophørsskemaer (OPUS) DREAM-data vedr. beskæftigelseseffekter Erhvervsregisterdata vedr. iværksætteri 9 Kapitel 10 svarer således delvist til kravene i COM: Information extraction grid for the synthesis of final evaluation, C: Mainstreaming equal opportunities for women and men.

9 Slutevaluering af Mål 3-programmet Side Forkortelser AF ARF EBST EES EPD ESF ETG FAQ IKT Arbejdsformidlingen Amtsrådsforeningen Erhvervs- og Byggestyrelsen European Employment Strategy (den europæiske beskæftigelsesstrategi) Enhedsprogrammeringsdokument Den Europæiske Socialfond Europæiske Tematiske Grupper Frequently Asked Questions Informations- og kommunikationsteknologi NAP National Action Plan (Danmarks nationale handlingsplan for beskæftigelse 2004) NAPincl NRP NTG PAE PGU POU PT RAR RSU TA-midler VEU National Action Plan Inclusion (Danmarks nationale handlingsplan til bekæmpelse af fattigdom og social udstødelse 2003/2005) National Reform Programme (afløser ultimo 2005 NAP) Nationale Tematiske Grupper Projektadministrerende enhed (både i EBST og regionalt i alle amter) Projektgodkendelsesudvalg Programovervågningsudvalg Programtillæg Det Regionale Arbejdsmarkedsråd Det Regionale Socialfondsudvalg Midler til teknisk assistance Voksen- og efteruddannelsesområdet

10 Slutevaluering af Mål 3-programmet Side 5 2. EXECUTIVE SUMMARY 2.1 Programme contents, context and implementation 10 The Objective 3 programme in Denmark has four priorities: 1) Active labour market policy; 2) Integration of weak unemployed persons; 3) Development of employee competences; and 4) Entrepreneurship. Each of these four priorities includes measures aimed at individuals as well as systems. The initiatives aimed at individuals are targeted at unemployed persons, persons in employment on the outskirts of the labour market and existing or potential entrepreneurs. The measures aimed at systems are targeted partly at the ordinary system for activation of unemployed persons, partly at enterprises, educational institutions etc. The evaluator finds that the Objective 3 programme has a successful strategic interaction with the European and national political framework within which it has to work; and in practice it is placed in the cross field between several different ordinary initiatives. An analysis of the financial weight of the Objective 3 programme shows, however, that the initiative under the Objective 3 programme makes up only a very limited part of the total Danish expenditure for job creation and growth. In 2003 the expenditure for active benefits totalled DKK 31,346m, compared with which the Objective 3 programme s annual budget of between DKK 416m and 531m, corresponding to between 1.3 and 1.7 per cent of the annual expenditure for active benefits, is rather insignificant. If expenditure for other relevant ordinary initiatives is included, the relative financial weight of the Objective 3 programme obviously seems even more limited. This limited financial weight also shows if the priorities and measures of the Objective 3 programme are analysed separately. However, priority 4 differs from the general pattern in that it makes up almost 15 per cent of the total registered expenditure for promotion of entrepreneurship in Denmark. The very limited financial weight of the programme compared with the expenditure for these ordinary initiatives means, however, that an effective mainstreaming of the results and experience generated by the Objective 3 projects is a prerequisite for creating actual added value. On top of this the Objective 3 programme in Denmark has not been fully implemented yet. At the time of final evaluation the most recent statement (including the second period of 2005) indicates that a total of DKK 1,399,978, have been spent for the total Objective 3 programme. This corresponds to 57.9 per cent of the indicative framework for the period , or 49.1 per cent of the indicative framework for the entire programme period For an elaboration of the results in this section, cf. chapters 4 and 6 below.

11 Slutevaluering af Mål 3-programmet Side Thus, a significant part of the funds under the Objective 3 programmes still have not been spent, and there is a risk of encountering N+2 problems on conclusion of the programme. Part of the under-consumption is due to a delayed programme start. The funds spent have generated a total of 2,372 project grants, of which 1,900 grants or 80 per cent have been given to projects under measures aimed at individuals. These projects have a total of 70,833 participants, of which 58,030 or 82 per cent have participated in projects under measures aimed at individuals. There is a fairly even distribution of men and women among the project participants per cent of the participants in projects under the measures aimed at individuals are women, and 53.6 per cent are women in projects under the measures aimed at systems. With a view to promoting the programme implementation, efforts have been made to relieve some of the problems previously caused by the actual implementation system. The prerequisites for the Programme Monitoring Committee s (PMC) work have thus been improved, and various information and mainstreaming activities have been initiated with a view to increasing the financial flow. And, measured in output there is a certain progress in the programme. 2.2 Programme outcome and impact Long-term effects of the total Objective 3 programme The final evaluation has had particular focus on the results (outcome and impact) generated by the Objective 3 programme in the medium and the long term and on the target completion which can be expected for the programme at present. The overall conclusion on outcome of the programme as a whole is that mainstreaming of innovative project ideas forms a relatively significant part of all projects together, i.e. those aimed at individuals as well as at systems. The primary innovative elements of all projects together concern the combination of project purpose and target group, project purpose and tools/initiative and project target group and tools/initiative. It is significant for the projects under the measures aimed at systems that they experiment more with involving external cooperation partners and with the distribution of roles among managers and employees. Special innovative elements in the projects aimed at individuals, however, concern the duration of the courses and the involvement of the target group. This difference in innovation focus is just what could be expected, given the differences in the purpose of measures aimed at individuals and systems, respectively, in the Objective 3 programme. It is thus the above innovative elements that are to a great extent sought mainstreamed by all Objective 3 projects together. Nevertheless, the effects of this mainstreaming are most visible among other caseworkers and at the administrative management level. As for the vertical mainstreaming to political decision-maker level, there is still room for improvement. 11 For an elaboration of the results in this section, cf. chapters 7, 8 and 10 below.

12 Slutevaluering af Mål 3-programmet Side 7 The fact that the mainstreaming success of the projects is primarily horizontal in relation to caseworkers and administrative leaders and only to a more limited extent vertical in relation to the political level thus also has immediate significance for the impact generated so far by the projects aimed at systems. It is thus still difficult to establish any major influence on and added value of the Objective 3 programme for the ordinary initiative. The impact generated by the projects aimed at individuals primarily concerns competence development and empowerment of the project participants and thus improved prerequisites for their getting into employment or maintaining their attachment to the labour market. These effects can be seen from the project participants replies to the final questionnaires as well as the project managers replies to the questionnaire. On the other hand, it is more difficult to establish outright employment effects. In the final evaluation such employment effects have been measured in a number of special runs in the DREAM register of the Danish Labour Market Authority which contains information on unemployment courses. Employment effects are calculated as the difference between the degree to which the unemployed persons have received public benefits for a period of 104 weeks prior to project conclusion until 13 and 26 weeks, respectively, after project conclusion. These special runs indicate that for the total Objective 3 measures aimed at individuals, the employment effects are either negative or positive, being very close to neutral. Please note that the DREAM run primarily makes sense as an effect goal for measures 2.1.A and 2.1.B, which are aimed at integrating weak unemployed persons in the labour market. The preceding unemployment period and public dependency ratio for this group do not artificially deteriorate the employment effect measured. For measures 2.1.A and 2.1.B the employment effect is on the whole neutral. This means that the Objective 3 initiative towards the target groups of these measures does not contribute to any major degree to their immediate integration in the labour market. Further, the DREAM run can say something about the effects of measure 3.2, which concerns retention of persons in employment. A positive or neutral employment effect would be a sign of success in retaining employed project participants in employment. For this measure the relatively low degree of public dependency increases slightly after project participation, meaning that measure 3.2 as well has a slightly negative, but close to neutral, employment effect. 12 Furthermore, the employment effects of similar ordinary initiatives are better than the employment effects of Objective 3 projects. A number of special runs for control groups for measures 1.1.A, 2.1.A and 2.1.B as well as a review of effect studies for related initiatives in relation to all the measures aimed at individuals indicate that the ordinary initiative typically has better direct employment effects. 12 For priorities 1 and 4 the DREAM run should far from stand alone as an effect goal, which is why the results are not referred to here.

13 Slutevaluering af Mål 3-programmet Side 8 However, the employment effects are probably relatively poor because the Objective 3 projects experiment with new types of initiatives and courses. Further, the effect measurements made indicate that the time aspect the period from conclusion of a course until effect measurement has significant influence on which conclusion to be drawn from the effects of the course/initiative. Therefore, it is interesting per se that the projects have generated relatively good empowerment and competence development effects among the project participants, which justifies an expectation that these improved prerequisites for employment in the long term will have actual employment effects Special focus on long-term effects of priorities 3 and 4 In continuation of the above general evaluation of the total impact of the Objective 3 programme there is special focus on the long-term effects of priorities 3 and 4. These priorities have been chosen for separate analysis because they have a special future relevance in relation to the current focus on growth drivers etc. Priorities 1 and 2 of the Objective 3 programme have thus been most relevant in the beginning of the programme period, whereas priorities 3 and 4 are found to have relatively more relevance at present and in future. For one thing, this relevance consideration is made because of the way in which the Objective 3 programme supports the re-launched Lisbon strategy and the government s Growth on Purpose growth strategy in respect of strategy as well as results. The overall conclusion on the underlying measures aimed at individuals of these priorities is that they have contributed positively to retention on the labour market (measure 3.2) and entrepreneurship (measures 4.1.A and 4.1.B), and that they have had positive effects on the competence development, empowerment and motivation of the project participants. Further, a longer time perspective is expected to reveal more nuances in the direct effects of these measures aimed at individuals. Thus the assessment of the effects aimed directly at individuals for priorities 3 and 4 are slightly more positive than for the evaluation of the total Objective 3 programme. As for the more indirect effects on system level of measures 3.2, 4.1.A and 4.1.B aimed at individuals, these are primarily visible at caseworker level in the same organisation as the project or other similar organisations. Only for measure 3.2 this corresponds to the declared aim of the measure. Considering the underlying measures aimed at systems (3.1.A, 3.1.B, 3.3 and 4.1.C), the overall conclusion is that it is difficult based on the existing data material to find visible longterm effects at political level. For measures 3.1.A and 3.1.B it is not possible at present to establish any close coordination between the Objective 3 projects and related monitoring initiatives, and it is still not clear whether the projects have caused cohesion and synergy in the analyses. Measures 3.3 and 4.1.C as well have primarily been successful with effects at practical-administrative level internally in the project organisation or in other organisations at the same level as the project. There are variations between the measures, both in focus on mainstreaming to political level and in the degree of financial implementation.

14 Slutevaluering af Mål 3-programmet Side Prerequisites for increased outcome and impact The overall the conclusion on the Objective 3 programme as a whole as well as on priorities 3 and 4 is that all priorities and priority 4 in particular have great potential for achieving political significance. As said, the limited financial weight of the Objective 3 programme and its not yet completed implementation mean, however, that a significant mainstreaming initiative is required before this added value potential can be fully realised. No matter if we consider the Objective 3 programme as a whole or focus specifically on priorities 3 and 4, it is clear that the mainstreaming initiative put into practice by the projects have primarily been successful in relation to the caseworker level and that the lack of mainstreaming to and visibility of the results of the projects at political system level illustrate that the otherwise great potential political significance of the measures has not yet been fully realised. In future, mainstreaming thus should be given high priority among all Objective 3 projects and in the programme administration. Add to this, however, the assessment that the implementation system for the Social Fund Objective 3 programme in Denmark does not support the realisation of long-term system-aimed effects optimally. The evaluator therefore approves of the recently initiated central mainstreaming initiative from the NAEC, which means, among other things, increased information to the projects on their mainstreaming obligation, a systematised mainstreaming initiative across the project mass on the part of the NAEC and thus improved possibilities of PMC contributions to the vertical exchange of experience. In future, these initiatives will hopefully rectify the programme s lack of visibility at political level. It is further recommended that the PMC itself gives a higher priority to monitoring and seeks more knowledge on the actual results in the project mass, including via the systematised information that the NAEC will procure in future about the results of the project mass. Finally, the evaluator recommends that the implementation system s support of the effects aimed at systems is significantly improved in the coming programme period. 2.3 Added value in relation to EES 13 Finally, the added value of the Objective 3 programme has been assessed in relation to the implementation of the EES at European and national as well as local and individual level. Having established that the Objective 3 programme at the strategic level supports the EES objectives, it is thus interesting to assess which results the Objective 3 programme has provided. The conclusion is that the actual added value is most visible at the local and individual level. At local level the planning and implementation of the measures in relation to unemployed persons and persons in employment by municipalities, Danish Employment Services, enterprises and educational institutions the potential for added value in the Objective 3 programme is found in the fact that the implementation of an Objective 3 project takes place in project form. The positive gains of this are, first that project manager and participants have been withdrawn and thus released from the cultures and traditional thinking of the ordinary 13 For an elaboration on the results in this section, please see chapter 9.

15 Slutevaluering af Mål 3-programmet Side 10 system; and secondly that project managers are forced to define and think in development goals rather than operation. Among these players by far the best results of the mainstreaming initiatives of the projects are found. As said, the added value at the individual level consists in the competence development and empowerment that the project participants experience via the Objective 3 programme. 2.4 Follow-up on the mid-term evaluation and recommendations for the future 14 On the whole the evaluation of the Objective 3 programme in Denmark has continuously given rise to a number of recommendations for improvement of the ongoing programme. Further, the evaluations have given rise to a number of recommendations for the future which can profitably be implemented in connection with the preparation of the coming programme period. Table 2.1: Follow-up on the mid-term evaluation five main challenges Main challenge 1. Experience gathering and dissemination of knowledge: In 2003 a more coordinated experience gathering and dissemination of knowledge were called for, partly across counties and agency in relation to influencing the ordinary initiatives (horizontally), partly centrally in relation to influencing the national strategies and reform initiatives (vertically). Status on follow-up At the end of 2005 the evaluator has found an increased initiative and great progress in ensuring mainstreaming, dissemination of knowledge and experience gathering from the Objective 3 programme since The improved initiative concerns the horizontal as well as the vertical mainstreaming at programme as well as project level. However, the results of the mainstreaming are so far most clear for the horizontal mainstreaming at project level. The follow-up on the mid-term evaluation recommendations can be summed up in the following initiatives: At programme level a number of initiatives have been implemented in the course of 2004 and 2005 to increase the awareness of the Objective 3 programme. A revised and integrated information strategy has been drawn up, and have been updated, and thematised application rounds have been conducted in 2005 in cooperation with a number of counties. In 2005 an extensive central mainstreaming initiative has further been initiated. The NAEC has developed the internal dissemination of knowledge and experience gathering in the Objective 3 administration across the regions and across the central and regional levels. At project level a marked improvement in the work on sharing concrete experiences of methods and specific initiatives has taken place in that an increasing number of projects have carried out mainstreaming and information activities in the course of the year. The need for this has been very significant, as the experience from more than 1,000 projects should preferably be disseminated and made accessible. The homepage makes homepages available to all projects where the projects can describe contents and communicate experience gained. However, despite an effort from the NAEC as well as a number of counties, almost half the projects still do not make use of this opportunity to share knowledge. If more projects were to update these homepages, the experience gathering would be benefit considerably from such a database source. At the same time there is still reason to stress the necessity of working on gathering, embedding and disseminating the experience from the programme among the players in charge of the ordinary initiative. 2. Coordination with the other national strategies: In 2003 the evaluator called for increased coordination At the end of 2005 the evaluator has found that since 2003 a number of initiatives have been implemented with the purpose of increasing coordination. A marked example is the turning of the programme in a 14 For an elaboration of the results in this section, please see chapter 5.

16 Slutevaluering af Mål 3-programmet Side 11 Main challenge of the Objective 3 programme with other national strategies. Status on follow-up more growth-oriented direction via reprogramming with focus on priorities 3 and 4 as well as implementation of thematised application rounds that relate to the government s growth strategy. Also, an increased cooperation between the social fund administrations of Objective 2, Objective 3 and EQUAL and the setting up of a number of internal work groups with focus on achieving synergy between the Objective 2 and Objective 3 programmes in particular result in increased integration between these programme areas and a strengthening of the connections between the overall strategies and respective objects of the funds. At the same time it has been a priority that the Social Fund be made visible as a continuation of the Regional Growth Strategy and not only in relation to the national action plans in NAP and NAPincl. Finally, the NAEC has sought to increase the interaction with several of the contact persons from the different ministries and representatives in the PMC. A concrete result of the interaction is the drawing up of a recommendation on EU tenders on global grants. A number of initiatives have thus been implemented to ensure coordination in relation to the other national strategies. However, much work is still needed in this respect, not least in the light of the need to increase consumption in the Objective 3 programme in the remaining period. Therefore, main challenge no. 2 is still relevant 3. Administrative routines across counties and between board and counties: In 2003 the evaluator called for a higher degree of coordination of good administrative routines and procedures across counties and between the agency and counties, including common guidelines and coordination in relation to suggestions for themes for applications, procedures for application assessment, application approval, administration, audit, control and project evaluation. Since 2003 a large number of improvement initiatives have been implemented with the purpose of strengthening the coordination of good administrative routines and procedures between the programme and project administrations across the 16 project administrations. One of the initiatives is the drawing up of an internal manual for project administration, which has since then been supplemented with a caseworker guide that also includes comprehensive instructions in connection with status reporting. Further, the development of a new audit and control model has been on the way together with a guide in handling cases of fraud. Finally, quality assurance has been made on the drawing up of annual report and the secretariat services to the PAC and the PMC. The development of a number of indicators for the drawing up of benchmarking and identification of good practice by the 16 PAU s was started in 2004 but has not yet been finished. Taken together, a large number of initiatives have been initiated to ensure a higher degree of uniformity across the 16 administrations. It is found particularly valuable that a number of procedures have been drawn up centrally, which are subsequently accessible to the remaining part of the Objective 3 administration. However, there is still a need for regional embedding of the initiatives started. The consumption situation for the entire programme further indicates that there will be a great need for the next two years for an even more efficient and flexible organisation. The evaluator continues to recommend the highest degree of coordination of good administrative routines and procedures across counties and between the agency and counties, particularly concerning common guidelines and coordination in relation to suggestions for themes for applications, procedures for application assessment, application approval, administration, audit, control and project evaluation. For a coming programme period these issues are expected to change their nature, as the decentralised project administrations cease to exist. 4. Further development of indicators: In 2003 the evaluator called for further development of indicators for frameworking as well as measurement of project results. The EU Commission as well as the NAEC and the evaluator require development and use of constant and valid indicators. It has, however, been difficult to finish the work on developing indicators. At the turn of the year the NAEC works on laying down indicators for use in the coming programme period As the work on indicators has not been

17 Slutevaluering af Mål 3-programmet Side 12 Main challenge 5. Strengthening of the PMC s work: In 2003 the evaluator called for a strengthening of the PMC s work. Status on follow-up concluded, the evaluator finds that this main challenge maintains its clear relevance. There has been broad agreement that this challenge is very important and a necessary prerequisite for being able to meet other main challenges. The NAEC has thus worked on strengthening the PMC through laying down of rules of procedure, participants, meeting frequency and general level of involvement and information as well as periodic reporting of programme status. At the same time the distribution of competence between the PMC and PAC has been specified. These initiatives have ensured that the work which is based in the PMC has become more controlling and significant to the implementation and direction of the programme than before. In practice this has meant a clear revitalisation of the PMC s work, which will be important for the PMC to play an important role in the requisite completion of the programme in terms of contents and finances during this period. The overall conclusion is that the NAEC has worked very focused and systematic on meeting most recommendations which are a result of the evaluations conducted and thus the many inputs which changing players have given the evaluator during the period from 2002 to As regards the preparation of the coming programme period, the main recommendations are summed up in the below table: Table 2.2: Recommendations for the coming programme period Recommendations for the coming programme period That efforts are made to avoid a similarly delayed programme start in the coming period, as such delay will be difficult to catch up with That in case of delayed programme start a prolonged time span in which to use ESF funds (N+3 or the like) is established for the first part of the programme period with a view to relieving any flow problems that originate from the delay That the administrative set-up concerning the administration of the Social Fund in Denmark is continuously sought simplified to make it easier for all parties involved to work with That a co-financing system is established, which builds more on Finance Act funds or co-financing from other pools than on the benefits or pay of the participants, with a view to reducing the administrative burden and providing the projects with a more secure basis of financing and thus increase the effective project implementation with increased focus on contents and effect 15 That greater flexibility is ensured in the Danish programming than in the current period to facilitate a possible re-programming in case of changed needs That the programming of the future programme increases the interaction between the Social Fund and other national strategies and framework conditions of growth That focused tenders and announcements are increasingly used to ensure the desired direction of the programme That a faster and more efficient annulment of funds not spent is ensured with a view to improved cash management and the possibility of fast renewal of grants 15 The Danish co-financing system, which is based on participant financing, was created in the 1980s in a narrow cooperation between the Danish Ministry of Finance, the Danish Ministry of Employment and the Danish Ministry of Social Affairs as the politically feasible co-financing. The evaluator is aware that the existing co-financing system is neutral for the government in terms of costs. However, the evaluator maintains its recommendation to reconsider the model in the light of the existing model causing a risk of N+2 problems and decommitment.

18 Slutevaluering af Mål 3-programmet Side 13 That a central and focused information and mainstreaming strategy is planned and integrated in the programme and that the implementation of this strategy is given much focus from programme start That the future programme uses Danish formulation in frameworking the programme That gender mainstreaming is incorporated in the future programme, and that the formulation of the gender mainstreaming aspect and indicators hereof is adapted to a Danish context That empowerment is also incorporated in the future programme. It may be considered to introduce specific support for the preparation phase with a view to ensuring increased and earlier involvement of the participants That the administrative management and reporting system is reconsidered with a view to improving reporting, monitoring, control and evaluation That relevant, measurable indicators are developed with a view to a more precise assessment of programme implementation and effects That the identified quality indicators for the good project are used when selecting and approving projects

19 Slutevaluering af Mål 3-programmet Side SAMMENFATNING 3.1 Programmets indhold, kontekst og implementering 16 Mål 3-programmet i Danmark er udmøntet i fire prioriteter med overskrifterne 1) Aktiv arbejdsmarkedspolitik, 2) Integration af svage ledige, 3) Kompetenceudvikling af ansatte, samt 4) Iværksætteri. Under hver af disse fire prioriteter findes såvel individ- som systemrettede foranstaltninger. De individrettede foranstaltninger retter sig mod hhv. ledige, beskæftigede på kanten af arbejdsmarkedet og eksisterende eller potentielle iværksættere. De systemrettede foranstaltninger er målrettet dels det ordinære system for aktivering af ledige, dels virksomheder, uddannelsesinstitutioner, mv. Mål 3-programmet vurderes at have et vellykket strategisk samspil med den europæiske og nationale politiske ramme, hvor inden for det skal fungere, og det placerer sig i praksis i krydsfeltet mellem flere forskellige ordinære indsatser. En analyse af Mål 3-programmets finansielle vægt viser dog, at indsatsen under Mål 3-programmet kun udgør en meget begrænset del af de samlede danske udgifter inden for beskæftigelsesfremme og vækst. I 2003 udgjorde udgifterne til aktive ydelser alene mio. kr., i forhold til hvilket Mål 3- programmets årlige budget på mellem 416 og 531 mio. kr., svarende til mellem 1,3 og 1,7 % af de årlige udgifter til aktive ydelser, ikke syner af meget. Medtages udgifter til øvrige relevante ordinære indsatser, bliver Mål 3-programmets relative finansielle vægt selvsagt endnu mere begrænset. Denne begrænsede finansielle vægt går igen, hvis Mål 3-programmets prioriteter og foranstaltninger analyseres separat. Dog adskiller prioritet 4 sig fra det generelle mønster ved at udgøre næsten 15 % af de samlede registrerede udgifter til iværksætterfremme i Danmark. Programmets stærkt begrænsede finansielle vægt sammenlignet med udgifterne til de ordinære indsatser betyder, at en effektiv mainstreaming af de resultater og erfaringer Mål 3- projekterne genererer, er en forudsætning for at skabe reel merværdi. Hertil kommer, at Mål 3-programmet i Danmark endnu ikke er fuldt ud implementeret. På slutevalueringstidspunktet viser den seneste opgørelse (inkl. 2. periode 2005), at der for det samlede Mål 3-program er forbrugt kr ,92. Dette svarer til 57,9 % af den indikative ramme for perioden , eller 49,1 % af den indikative ramme for hele programperioden Således er en væsentlig del af midlerne under Mål 3-programmet endnu uforbrugte, og der er risi- 16 For en uddybning af resultaterne i dette afsnit, se kapitel 4 og 6 nedenfor.

20 Slutevaluering af Mål 3-programmet Side 15 ko for N+2-problemer i forbindelse med programmets afslutning. En del af underforbruget skyldes en forsinket programstart. De forbrugte midler har genereret i alt projektbevillinger, hvoraf de bevillinger - eller 80 % - er givet til projekter under individrettede foranstaltninger. Disse projekter tæller tilsammen hele deltagere, hvoraf eller 82 % - har deltaget i projekter under individrettede foranstaltninger. Kønsfordelingen blandt projektdeltagerne er nogenlunde lige, således at der er 50,3 % kvinder i projekter under de individrettede foranstaltninger og 53,6 % kvinder i projekter under de systemrettede foranstaltninger. Med henblik på at fremme programimplementeringen er der gjort en indsats for at udbedre nogle af de problemer, selve implementeringssystemet tidligere indebar. Således er forudsætningerne for programovervågningsudvalgets arbejde forbedret, ligesom forskellige informations- og mainstreamingaktiviteter er iværksat med henblik på at øge det finansielle afløb. Målt på output ses da også en vis fremdrift i programmet. 3.2 Programmets outcome og impact Langsigtede effekter af det samlede Mål 3-program Fokus i slutevalueringen har i særligt høj grad været på de resultater (outcome og impact), Mål 3-programmet har genereret på mellemlang og lang sigt, samt den målopfyldelse, programmet på nuværende tidspunkt kan forventes at få. Den helt overordnede konklusion vedr. outcome af programmet som helhed er, at mainstreaming af innovative projektideer indgår som en forholdsvis væsentlig del af alle projekter under ét, dvs. både individ- og systemrettede. De primære innovative elementer i alle projekter under ét vedrører kombinationen af projektets formål og målgruppe, projektets formål og redskaber/indsats samt projektets målgruppe og redskaber/indsats. Kendetegnende for projekterne under de systemrettede foranstaltninger er, at de i højere grad eksperimenterer med inddragelse af eksterne samarbejdspartnere og rollefordeling blandt ledere og medarbejdere. Særskilte innovative elementer i de individrettede projekter vedrører derimod forløbenes varighed og inddragelse af målgruppen. Denne forskel i innovationsfokus er netop, hvad man kunne forvente, givet forskellene i hensigten med hhv. individ- og systemrettede foranstaltninger i Mål 3-programmet. Det er altså ovenstående innovative elementer der i høj grad søges mainstreamet af alle Mål 3- projekterne under ét. Ikke desto mindre er effekterne af denne mainstreaming mest synlig blandt andre praktikere samt på det administrative ledelsesniveau. Hvad angår den vertikale mainstreaming til politisk beslutningstagerniveau er der fortsat rum for forbedring. Det forhold, at projekternes mainstreamingsucces primært ligger horisontalt i forhold til praktikere og administrative ledere, og kun i mere begrænset omfang vertikalt i forhold til det politiske niveau, har således også direkte betydning for den impact, de systemrettede projekter 17 For en uddybning af resultaterne i dette afsnit, se kapitel 7, 8 og 10 nedenfor.

21 Slutevaluering af Mål 3-programmet Side 16 indtil videre har kunnet generere. Således er det endnu svært at konstatere nogen omfattende indflydelse på og merværdi af Mål 3-programmet på den ordinære indsats. Den impact, de individrettede projekter genererer, drejer sig først og fremmest om kompetenceudvikling og empowerment af projektdeltagerne, og dermed forbedrede forudsætninger for at komme i beskæftigelse eller fastholde tilknytningen til arbejdsmarkedet. Disse effekter fremgår af såvel projektdeltagernes besvarelser af ophørsskemaer som projektledernes besvarelse af spørgeskemaet. Derimod er det sværere at konstatere deciderede beskæftigelseseffekter. Sådanne beskæftigelseseffekter er i slutevalueringen målt ved en række særkørsler i Arbejdsmarkedsstyrelsens DREAM-register, som indeholder oplysninger om ledighedsforløb. Beskæftigelseseffekter opgøres her som forskellen mellem graden af offentlig forsørgelse i en periode fra 104 uger før projektafslutning til hhv. 13 og 26 uger efter projektafslutning. Disse særkørsler viser, at der for Mål 3-programmets individrettede foranstaltninger under ét er tale om enten negative eller meget tæt på neutrale positive beskæftigelseseffekter af de individrettede foranstaltninger. Det bemærkes, at DREAM-kørslen primært giver mening som resultatmål for foranstaltning 2.1.A og 2.1.B, som er rettet mod integration af svage ledige på arbejdsmarkedet. For denne gruppe gælder, at deres forudgående ledighedsperiode og offentlige forsørgelsesgrad ikke kunstigt forringer den målte beskæftigelseseffekt. For foranstaltning 2.1.A og 2.1.B er beskæftigelseseffekten stort set neutral, svarende til at Mål 3-indsatsen over for disse foranstaltningers målgrupper ikke ændrer nævneværdigt ved deres umiddelbare integration på arbejdsmarkedet. Endvidere kan DREAM-kørslen sige noget om effekterne af foranstaltning 3.2, som vedrører fastholdelse af beskæftigede. Her vil en positiv eller neutral beskæftigelseseffekt være tegn på succes med at fastholde de ansatte projektdeltagere i beskæftigelse. For denne foranstaltning stiger denne relativt lave grad af offentlig forsørgelse lidt efter projektdeltagelse, svarende til, at også foranstaltning 3.2 har en svagt negativ, men tæt på neutral beskæftigelseseffekt 18. Endvidere er beskæftigelseseffekterne af tilsvarende ordinære indsatser bedre end Mål 3- projekters. Således viser en række særkørsler for kontrolgrupper til foranstaltning 1.1.A, 2.1.A og 2.1.B samt en gennemgang af effektstudier for beslægtede indsatser i forhold til alle de individrettede foranstaltninger, at den ordinære indsats typisk har bedre direkte beskæftigelseseffekter. Det skal imidlertid bemærkes, at de forholdsvis ringe beskæftigelseseffekter formentlig hænger sammen med, at Mål 3-projekterne netop eksperimenterer med nye typer indsatser og forløb. Endvidere viser de foretagne effektmålinger, at tidsaspektet, forstået som perioden fra afslutning af forløb til effektmåling, har væsentlig indflydelse på, hvilken konklusion man kan drage om forløbets/indsatsens effekter. Derfor er det i sig selv interessant, at projekterne har 18 For prioritet 1 og 4 bør DREAM-kørslen langt fra stå alene som effektmål, hvorfor resultaterne heller ikke er refereret her.