FINDINGS IN THE NEIS EVALUATION
Report prepared for the:
DEPARTMENT OF EMPLOYMENT, WORK PLACE RELATIONS AND SMALL BUSINESS
the CENTRE for LABOUR MARKET RESEARCH
Western Australia, 6150
TABLE OF CONTENTS
This report presents the findings of an evaluation of NEIS and a set of recommendations based on deficiencies identified through the process of the evaluation. The report also responds to each of the terms of reference and makes recommendations where appropriate.
The report has reviewed international and Australian literature on evaluations of self-employment programmes, as well as providing summaries of previous evaluations of NEIS. An analysis of participant outcomes data, a telephone survey of 350 participants and all NEIS providers was undertaken and the findings are presented. Stakeholder interviews were conducted and included representatives from the DEWRSB, peak bodies representing industry viewpoints on the scheme as well as employment service providers. Providers and participants were also interviewed in-depth, and issues raised from this stage of the evaluation are presented in the report. The report concludes with an analysis of findings against the terms of reference.
A number of key recommendations are made in relation to issues specified in the terms of reference as well as with respect to issues uncovered through the respective surveys.
Milestones should reflect the seasonality of regional economies. This can be achieved by allowing providers to specify their milestones in the contract or through an assessment by DEWRSB of previous regional commencement patterns.
The economic circumstances of regions may have a large bearing on provider performance. Therefore, provider performance against all KPI measures should be weighted by regional specific factors.
Quotas should be set for disadvantaged clients. A Key Performance Indicator (KPI) measuring provider performance (ie commencements) against their quota of places for disadvantaged clients should be introduced.
KPI 2 measures 3 and 12 month off-benefit outcomes. Outcomes at 12 months after provider obligations cease are most likely beyond the influence of providers and affected by a myriad of other factors. Therefore KPI 2 should only consider 3-month outcomes. An alternative is to extend provider services to participants for a further 12 months and monitor accordingly (ie retain KPI 2 in its current format). Providers would need to be funded to do this.
NEIS should be retained, but specifically targeted at the most disadvantaged clients.
More general assistance to micro and small business assistance along the lines of NEIS should come under a small business or industry portfolio, not a labour market portfolio. The feasibility of utilising components of existing NEIS infrastructure, such as providers and contract management, should be examined.
The following provides a brief overview of what the main findings of the evaluation:
Self-employment schemes usually fall into one of two categories- they are either aimed at stimulating the small business sector, or are aimed at getting people out of unemployment and into work. All European Union (EU) programmes were essentially of the latter type, with eligibility usually restricted to unemployed people. In some instances eligibility is widened to include people at risk of job loss.
All member States of the EU introduced self-employment programmes at some stage over the last two decades. They generally are considered successful, although they account for very small shares of labour market programme expenditures.
One of the main issues addressed in evaluations of overseas programmes is deadweight loss1. Deadweight loss is highest for those programmes where the eligibility is widest. Schemes targeted specifically at clients who are disadvantaged have the lowest incidence of deadweight loss, the downside is that lower survival rates may result. However, in Denmark the scheme is focussed on the long-term unemployed and it has survival rates among the highest of any country.
Displacement effects2 are also considered, although there is fairly limited information on the true extent of displacement. What studies exist suggest that the extent of displacement is fairly minimal.
One of the major pitfalls of self-employment schemes is the prevalence of very low incomes among business operators.
Past evaluations of NEIS
NEIS was established as a pilot scheme in 1985 and evaluated in 1988. The evaluation of the pilot stage found that long-term unemployed had success rates that were almost identical to those who were not long-term unemployed.
One job for every 1.5 surviving NEIS businesses was created.
The first major evaluation of NEIS took place in 1993, this revealed that 64 per cent of NEIS participants were still in business 3 months after cessation of their allowance. A further 9 per cent were in employment. For every 10 NEIS businesses, there were 1 full-time and 4 part-time jobs created. Survival rates dropped to 54 per cent 12 months after cessation, with 9 per cent still in other employment.
Although participants who had been long-term unemployed prior to participating in NEIS had similar self-employment outcomes, their incomes were substantially lower.
Despite apparent high success rates, NEIS was a costly programme when compared to other forms of assistance provided at that time.
A 1995 evaluation of NEIS based on Post-Programme Monitoring surveys found that less than 1 per cent of eligible job seekers actually participate in NEIS.
Employment outcomes for participants who had been long-term unemployed prior to being in NEIS were only slightly lower than for those who were previously short-term unemployed.
Participants rated mentors quite highly with 93 per cent stating that NEIS had improved their chances of successfully operating a small business.
Analysis of Departmental data
An analysis of Departmental data for May 1998 to February 2000, was undertaken. The analysis includes an econometric assessment of the determinants of outcomes. The results of the analysis show that:
Around 80 per cent are in some form of employment 3 months after the cessation of NEIS allowance. Of these, 65.9 per cent are in self-employment, 9.1 per cent are in full-time employment and 5.8 per cent are in part-time employment. 12.5 per cent are unemployed and 6.7 per cent are out of the labour force.
Evaluation of past programmes run by DEETYA with matched control groups suggest about 22 per cent of clients would have found work without participating in any form of assistance. As NEIS is not closely targeted this figure would be in excess of 22 per cent.
Outcomes do not vary greatly between disadvantage groups, such as sole parents, people with a non-English speaking background and people from an Aboriginal and Torres-Strait Islander decent.
Outcomes vary predictably by educational attainment, with the more highly qualified achieving outcomes that are 15 percentage points higher than those with the lowest level of schooling.
Persons aged over 55 years have markedly lower probabilities of being in employment or exiting benefits.
Results from the multivariate analysis show that males are more likely to achieve positive outcomes than females, while persons with disabilities and from non-English speaking backgrounds are less likely to be in employment. Controlling for other factors, the probability of being in employment increases with the level of education and the effect is particularly strong for people with degrees or higher.
There is a strong negative effect on outcomes for those participants who were very long-term unemployed (greater than 2 years) prior to commencing NEIS.
Off-benefit outcomes are affected by the economic conditions in the State that the participant resides.
There is a high level of satisfaction with the four main service components of NEIS, especially the training component. The tailoring of assistance to individual needs is regarded less favourably.
NESB clients reported higher satisfaction levels for all facets of NEIS, while long-term unemployed clients had lower levels of satisfaction with the overall quality of assistance and service.
There is no marked pattern of satisfaction levels with age.
An extra 488 jobs were created from 2 567 placements over the period - 131 full-time and 357 part-time/casual.
For every 100 NEIS commencements there are 81 in either self-employment or other employment and 19 secondary jobs (created by the NEIS businesses). Not all of these jobs are net additions to employment, as some of them result from the displacement of other jobs. Also, many would have found employment without NEIS assistance (deadweight loss).
On the basis of 3 month post-NEIS employment outcomes, the gross cost per positive employment outcome for NEIS clients is $13 547, and $10 965 per employment outcome when secondary jobs are taken into account.
These figures understate the true cost (ie. the net cost) as they do not take into account deadweight loss or displacement effects. Taking these into account suggests the true cost could be in excess of $30 000 per employment outcome and $20 000 if secondary employment is included.
Given that NEIS is not closely targeted at disadvantaged job seekers it is an expensive way of assisting participants into work.
Survey of Participants
Although NEIS may be a useful avenue for disadvantaged job seekers, it is a relatively small programme and only very small numbers are offered places.
87 per cent of NEIS participants over 50 years of age said they would have had some difficulty finding employment or would not have found work at all without NEIS. In comparison, only 28 per cent of 18-34 year olds said they would have found it difficult.
52 per cent of participants surveyed are self-employed. 26 per cent are not in paid employment, 22 per cent are employees.
12 months after cessation of NEIS allowance there are at most 73 per cent of participants are in self-employment. This drops to 63 per cent 9 months after NEIS finishes, with 57 per cent still in self-employment 12 months post-NEIS assistance. The survival rate continues to decline after 1 year, dropping to 52 per cent 18 months after benefit ceases.
43 per cent of secondary employment goes to disadvantaged groups (ie. less than 19 years old, over 45 years old, NESB and ATSI).
About 47 per cent of survivors and non-survivors have to raise capital to start their businesses. However, the amount raised by surviving businesses is substantially higher than non-surviving businesses.
The 3 principal sources of funds are savings, loans from credit providers (such as banks) and loans from family and friends. Survivors, on average, raise up to 8 time more than non-survivors from savings and 2.3 times more from credit providers.
2 in every 3 participants do not to put further funds into their business once it is operating. Of these, 11 per cent attribute the reason to not being able to get additional finance. Most of these are participants who are no longer running their business.
24 per cent of survivors and 38 per cent of non-survivors said that they would have preferred their allowance as an up-front lump-sum payment.
62 per cent of all participants said they would have started a business without NEIS assistance- 73 per cent of survivors and 54 per cent of non-survivors. Evaluation of similar schemes in the UK record about 40 per cent.
60 per cent of survivors and 45 per cent of non-survivors would still have started their business if the NEIS allowance were lower.
The average net income for all participants was $438 per week, the median was $120. The corresponding figures for survivors were $780 and $250 and for non-survivors $179 and $25 respectively.
73 per cent of non-survivors were earning $300 or less just before they ceased trading. 52 per cent of surviving business are currently earning less than this.
About half of non-survivors end up with less money than they put into their business. 23 per cent of non-survivors are now in debt as a result of having started a business under NEIS. The worst affected are young families and sole parents.
44 per cent of participants said that customers could have easily obtained their product or service elsewhere, suggesting that there is some degree of displacement.
Survey of providers
Most providers thought the participant eligibility criteria were appropriate.
39 per cent of providers thought that the business eligibility criteria were too tight.
Over half of all providers could not meet the demand for places.
About 20 per cent of providers were having difficulty in meeting their quota of places.
Virtually all providers said that NEISACs do not adversely affect outcomes.
Over half of all providers thought that the structure of payments was inappropriate, but there was no consensus on how they should be changed.
Most stakeholders felt that disadvantaged clients were well represented in NEIS.
There is widely held view among stakeholders NEIS should be targeted specifically at those who are most likely to succeed.
There was a consensus among stakeholders that eligibility criteria are appropriate, although there was some concern over the interpretation of the business eligibility criteria in some instances.
The difference between eligibility criteria for Self Employment Development Scheme (SEDS)3 and NEIS is difficult to justify.
The Integrated Employment System (IES) still has some flaws, particularly in relation to whether job seekers are accurately identified as being eligible to participate in the scheme.
There was some concern expressed over the quality and consistency of advisory committees (NEISACs). Similar complaints were made about mentors.
It was thought that the way milestones are applied in the contracts does not adequately reflect the seasonality of economic activity in some regions.
There was some criticism that NEIS is not adequately promoted. Also, Centrelink is not adequately informed about NEIS and, as a result, does not refer enough job seekers to the scheme.
It was felt that KPI 1 is too inflexible for providers in regional areas. KPI 4 measures providers on an outcome that is largely outside of their control.
Some stakeholders suggested that performance measures should include the additional employment created by participants.
All stakeholders were asked whether they thought that a lump-sum payment instead of the existing fortnightly payment would be beneficial. All rejected the approach, as it would be difficult for participants to manage their cash flow.