Job Network evaluation - Stage one implementation and market development
Correction Table 6.8
The data in Table 6.8 of Job Network evaluation Stage one: implementation and market development contained an error in the split between post assistance full-time and part-time employment status, which resulted in an overstatement of the full-time employment level. Data in this table were obtained from the department’s post program monitoring survey. The error resulted from a miscode of missing responses to the survey question on the number of hours worked. The internet version of the report has been updated to reflect this correction.
Executive summary
Background
In the 1996–97 Budget, the Federal Government announced a new framework for the delivery of labour market assistance. The new system was implemented on 1 May 1998.
The reforms involved:
- The integration of the key employment assistance functions, performed by the then separate Commonwealth Employment Service (CES) and Department of Social Security, to offer job seekers a single delivery point for accessing employment services, known as Centrelink.
- The development of a contestable market for publicly funded employment placement services, known as Job Network. This involved establishing a national tender process for the delivery of labour market services. Successful tenderers included government, private and community organisations.
- A continuum of assistance available to job seekers according to their level of disadvantage. Access to this assistance is mostly through Centrelink.
The Job Network represents a move to a much more flexible system for delivering employment assistance. In announcing these reforms in 1996, the Government’s four key objectives were to:
- Deliver a better quality of assistance to unemployed people, leading to better and more sustainable outcomes.
- Target assistance on the basis of need and capacity to benefit.
- Address the structural weaknesses and inefficiencies inherent in previous arrangements for labour market assistance, and to put into effect the lessons learnt from international and Australian experience of labour market assistance.
- Achieve better value for money.
To put the reforms into place, the Government changed the delivery mechanism for income support and cashed out a number of existing labour market programs. Job brokerage delivered through the CES was also replaced with an outsourced job-matching service.
The major changes introduced under Job Network included:
- The establishment of Centrelink in 1997 as the service delivery agency that offers self-service access to job search facilities, conducts assessments of labour market disadvantage using an enhanced profiling system (the Job Seeker Classification Instrument) and refers job seekers to Job Network members.
- Contestability in the delivery of employment services.
- Replacement of labour market programs and case management services with three key employment services, Job Matching, Job Search Training and Intensive Assistance; establishment of New Apprenticeships Centres, a service providing one-stop integrated support to employers; and the continuation of the New Enterprise Incentive Scheme (NEIS), which provides training and financial support to job seekers wanting to set up their own businesses.
- A Community Support Program for job seekers with severe barriers to employment that mainstream assistance services can not address.
- Project Contracting to provide employees for the seasonal harvests. This element of Job Network is not covered in this report.
When Job Network was implemented in May 1998, more than 300 private, community and government organisations began offering a range of employment services from over 1400 sites across Australia. This was a considerable increase in the number of sites that had offered employment services under the previous labour market assistance arrangements. At 30 June 1997 there were almost 700 outlets delivering labour market related services to job seekers, including 293 CES regional and branch offices and 321 contracted case management organisation outlets.
Evaluation methodology
This report is the first stage of an evaluation strategy endorsed by the Government in December 1996 and focuses on the implementation of Job Network and early market experience. The key issues for the overall evaluation are effectiveness, efficiency, quality of services and access. These criteria will be applied through the three stages of the evaluation. An additional evaluation issue—specific to Job Network—is market development, which carries potential impacts on all of these criteria.
The stage one report addresses access and quality of services. Only limited data on outcomes have been available, so this report does not examine the effectiveness of Job Network in securing sustainable employment for job seekers, nor the extent to which Job Network delivers value for money. This report covers the first 17 months of Job Network.
Main findings
Overall, stage one of the evaluation found that Job Network was implemented well and appears to be operating successfully. This was achieved in the context of large-scale changes to the provision of employment services. While it is too early to comprehensively measure the impact of Job Network on the employment prospects of job seekers, particularly disadvantaged client groups, early performance information in terms of commencements and placements, service quality and (job) outcomes are encouraging:
- In the first 17 months of Job Network, large numbers of job seekers have been assisted. Placements under Job Matching averaged 26 500 per month in the six months to September 1999. Commencements in Job Search Training and Intensive Assistance were around 4000 and 20 000 per month respectively.
- Quality of service to job seekers appears to have improved with the implementation of the Job Network. Job seekers rated the new arrangements as better at providing a professional service, paying attention to individual needs and in terms of being treated like a person rather than a number. The Job Network was also rated as better at improving job seeker chances of getting a job and at having a large range of suitable jobs. Employers rated Job Network as better than the CES on many aspects of service—in particular, for providing a quick, individually tailored service to employers and for showing an understanding of the commercial environment.
- A preliminary assessment of cost effectiveness is available for Job Search Training. Job outcomes compare favourably to those achieved by the former labour market program, Job Clubs, and training is delivered at about half the cost-per-job. Three-month post-assistance job outcomes were 37% for Job Search Training, compared to 25% for Job Clubs. The unit cost and cost-per-job outcomes for Job Search Training were estimated to be $418 and $1130 respectively. The corresponding costs for Job Clubs were $625 and $2500.
By the end of the first contract period, Job Network had made substantial progress towards a competitive market. Developments for the second tender round will ensure that this progress continues.
The evaluation has identified some areas where the Job Network’s operation could be improved. These centre mainly on equity of access to Job Network, particularly for indigenous job seekers, and in balancing quality service and flexible delivery. Choice-driven competition would benefit from further development of a culture of informed choice for employers and job seekers.
The evaluation also suggests that further work is required to assess whether the Intensive Assistance fee structure is clearly operating as an incentive to providers to pursue cost effectiveness and equity of outcomes for all job seekers accessing this service.
Detailed findings
Setting up the market
The transition to Job Network involved unprecedented changes to the delivery of labour market assistance. It included winding up the CES and cashing out most of the previous labour market programs, undertaking Australia’s largest ever employment services tender, setting up Centrelink and a corporatised public employment enterprise (Employment National) and communicating these extensive changes to the general public. At the same time, income support and services to job seekers and employers had to be maintained.
The probity adviser and the Australian National Audit Office reported favourably on the conduct of the tender process for setting up the market. The Audit Office findings also noted that measures were put in place to manage the risks to implementation from market failure and unfair advantage. In particular, the Audit Office concluded that the steps taken to reduce the risk of unfair advantage for potential tenderers and the lessons learnt from this tender process act as a guide for future contracting out exercises in the Australian Public Service.
Limited changes were made during Job Network’s first 17 months of operation, and these mainly affected Job Matching. The changes included the decision in August 1998 to open Job Matching to most job seekers not on income support.
Accessing Job Network
Job seeker satisfaction with the Centrelink registration process was 75% in March 1999, similar to the level a year earlier (78%). In qualitative research, a common view held by job seekers was that they did not fully understand what employment services were available and they reported frustration with the duplication of information required to access income support and employment services. Centrelink is currently developing an electronic form designed to support once-only collection of data. Young job seekers not in receipt of income support had a lower level of satisfaction with the registration process. It appears that the number of youth registering with Centrelink who are not eligible for income support declined with the introduction of Job Network. Centrelink staff and Job Network members interviewed in qualitative research were concerned that young job seekers’ contact with employment services had been reduced with the introduction of the Youth Allowance (at the same time as Job Network).
The Job Seeker Classification Instrument (JSCI) is used to identify job seekers for referral to Intensive Assistance. The profile of job seekers identified in need of Intensive Assistance or special needs assessment aligns closely with expectations. A post implementation review of the JSCI showed that the majority of job seekers were classified accurately through the JSCI. The review also identified areas for improvement and finetuning. Concerns with some aspects of the application of the JSCI were raised in a survey of Job Network members and in qualitative research for the evaluation. These need to be understood in the context of providers’ expectations of the JSCI. Concerns are being addressed through implementation of recommendations from the post implementation review.
In qualitative research, the computer systems (Integrated Employment System [IES] in particular, but also the Income Security Integrated System) were criticised by Job Network members and Centrelink staff. In relation to IES, there were criticisms of data quality and system functionality. While changes have been made, users felt the system could be further improved if it was made more user-friendly and allowed for easier input and extraction of information. Work on refining the IES functionality is continuing through the Job Network Information Technology Working Group and the Systems Integration Strategic Committee.
Job Matching
Job Network vacancies lodged each month on Australian Job Search have increased from 50 700 in September 1998 to 69 800 in September 1999.
Job Network was used by 38% of employers to lodge vacancies and recruit staff in the 12 months to June 1999—this compares to 32% of employers using the CES in 1997 (although it should be noted that at this time the CES was winding down). Job Network is the third most common recruitment method (the most common method being newspaper advertisements followed by head-hunting). The most common reason employers gave for not having used Job Network was that they were content with their current methods of recruitment and saw no need to change. Employers also reported that they had sufficient numbers of applicants approaching them and they did not need to use active recruitment methods.
Job placements through Job Matching reached almost 30 000 in September 1999. Seventy-eight per cent of these went to the initial Job Network eligible target group (that is, income-support recipients, participants in Community Development Employment Projects and youth), while 22% went to job seekers eligible for Job Matching only (that is, adult job seekers not receiving income support).
Overall, 80% of employers who used Job Network thought that the service and assistance they received was of high quality. Moreover, the group of employers using the Job Network includes many who were not recent users of the CES—60% of Job Network users had not used the CES in the year prior to its cessation.
Job Network was rated significantly better than the CES at sending job applicants who were interested in the job (70% compared to 46% under the CES in 1997). In spite of this improvement, 20% of employers thought that Job Network members did not send the type of applicants they needed.
The overall level of satisfaction with Job Matching services for all job seekers who have enrolled with Job Network members was 79%. The main reason for job seeker dissatisfaction with Job Matching services was a lack of sufficient assistance in finding employment (44% of those who were not satisfied). It should be noted that job seekers who are provided with self-help facilities often do not recognise these as a service.
Job Search Training
After completing Job Search Training, 76% of participants considered they had improved job prospects. Around 37% were employed three months after participation in Job Search Training.
Early data also suggest that Job Search Training is relatively cost effective. Job Search Training has similar objectives to the former labour market program, Job Clubs, but delivers training at about half the cost-per-job.
Recruitment of participants to Job Search Training has fallen short of expectations. At the end of September 1999, Job Search Training providers were operating at 74% of pro-rated contracted capacity. The introduction of the accelerated referral system has had a negligible effect on the level of commencements. Qualitative research suggests that commencements could be improved by Job Network members providing more information to job seekers and Centrelink. It was suggested that Job Search Training needs to be offered earlier while job seeker motivation is high and workplace experience is current. This has been factored into recent changes to eligibility criteria that halve the time before job seekers can volunteer for Job Search Training.
Intensive Assistance
Job seekers judge Intensive Assistance favourably. Eighty-one per cent of job seekers agreed that the service and assistance they received through Intensive Assistance was of high quality. Job seekers agreed that their provider was always available when they needed them (88%), gave the right kind of encouragement and support (83%), did all they could to help (85%) and took their special needs into account (79%).
Post program monitoring undertaken three months after leaving Intensive Assistance shows that for job seekers who left assistance up to the end of September 1999, around 43% were in a job (37%) or studying.
The early nature of these Intensive Assistance data and the fact that effectiveness measures have yet to be derived mean that comparisons with other forms of employment assistance should be treated cautiously. That said, early outcomes data for Intensive Assistance compare favourably to data for case management outcomes. Of job seekers who commenced Intensive Assistance between May and July 1998,23% were either off income support or on reduced-rate allowance 13 weeks after placement in employment or study, compared to 16% of job seekers commencing case management during 1995.
While there are reasons to be optimistic about the performance of Intensive Assistance, it is not clear that all the design features are functioning to the extent that may be possible. These include job seeker driven competition and the incentives in the fee structure:
- The feasibility of using job seeker choice as a means of improving provider performance was limited in the first contract period by a lack of comparative performance information given the establishment of a new market and the need to: encourage providers to invest the necessary resources; reduce the risk of provider failure; and ensure that job seekers participate where they do not choose a provider. Other mechanisms such as competitive tendering, service standards and quality audits contribute to improving provider performance.
- At this stage of the evaluation and in the early days of the market, it is not possible to establish whether the fee structure is clearly operating as an incentive to providers to pursue cost effectiveness and equity of outcomes for all job seekers.
Community Support Program
Between May 1998 and September 1999, 23 500 job seekers received a special needs assessment. Of these, 60% were subsequently referred to the Community Support Program and 15% were referred for Intensive Assistance.
While only a limited number of stakeholders have been consulted so far in the evaluation, most were positive about the Community Support Program. They considered the program to be of value because it allowed participants time and space to step away from the requirements of job seeking in order to address underlying barriers to employment. The main criticism of the program concerned its level of funding, which was considered insufficient to access some local services such as counselling or interpreter services.
New Enterprise Incentive Scheme
In the first 17 months of Job Network, more than 8800 job seekers received assistance to set up their own businesses through the New Enterprise Incentive Scheme (NEIS).
Results of the job seeker satisfaction survey showed that 93% of NEIS participants considered that participation in NEIS had improved their chances of successfully running their own business and 90% also felt that the services and assistance they had received were of a high quality.
New Apprenticeships
Apprentice and trainee commencements have increased from 60 000 in 1994–95 to 125 100 in 1997–98 and 189 600 in 1998–99. The number of apprentices and trainees in all age groups has increased with the greatest proportional increases in the 25 years and over group. Since 1996–97, there has been a rise in the proportion of females and a fall in the proportion of indigenous job seekers in apprenticeships and traineeships.
The industry groupings with the highest proportion of apprentices and trainees are wholesale and retail trade (20%), property and business services (18%) and manufacturing (14%).
Sixty-three per cent of employers, surveyed in June and July 1999, were not aware of New Apprenticeships Centres and 20% of employers who had used a New Apprenticeships Centre did not understand their role, suggesting the need for further marketing, which has now commenced.
Eighty-two per cent of employers who had recruited an apprentice or a trainee reported that they were satisfied with the service from New Apprenticeships Centres, while 13% said they were dissatisfied. Eighty-one per cent of employers said they would use the same centre again.
Breaching and compliance
The activity test is a means to enforce job search requirements and participation in active labour market assistance. The scope of this evaluation is to consider the role of Job Network members in the application of the activity test. A broader evaluation of the activity test is being undertaken by the Department of Family and Community Services.
Between July 1998 and August 1999, 97 700 activity test breach recommendations were made by Job Network members. Of these, Centrelink applied 43%. Breaches are not applied for a variety of reasons—the main ones being because the job seeker provided Centrelink with sufficient evidence to support their claim (34%) and insufficient documentation was provided by the Job Network member (13%). The rate of breach recommendations remained fairly constant (between eight and nine per 1000 job seekers) between October 1998 and May 1999. The rate was almost 14 per 1000 in August 1999. Most breach recommendations are because job seekers do not turn up to the provider they have been referred to.
Some Job Network members in qualitative research reported that they are reluctant to recommend a breach because of the administrative burden involved, the potential this has for interfering with the relationship they have developed with the job seeker and lack of feedback from Centrelink on the reasons breach recommendations are overturned. Regarding feedback, it should be noted that Privacy Act requirements restrict the amount of information that can be passed to Job Network members. Initiatives to address concerns about the administrative burden and complexity of breaching are underway. They include a joint agency study of the process at some Centrelink sites, seminars for Job Network members conducted by Centrelink and a quality assurance project.
Broader issues
Disadvantaged job seekers under Job Network
One of the key objectives of Job Network is to target effective assistance to job seekers on the basis of need and capacity to benefit. Given the extent of change, an important test of the success of formulating and implementing a system that will achieve this goal is to examine how disadvantaged groups have fared (job seekers from a non-English-speaking background, job seekers with a disability, young and older job seekers, and indigenous Australians).
Job seekers from a non-English-speaking background were generally more satisfied than other job seekers with the services from Centrelink and Job Network members. However, this group was somewhat more critical than other job seekers of the ability of Job Network members to take clients’ individual needs and circumstances into account. Further research should assess, in particular, whether it is appropriate to use the Community Support Program to address language or English literacy problems.
Satisfaction with Centrelink was above average for job seekers with a disability, while their satisfaction with all Job Network services was similar to the average. Selection rates for Intensive Assistance for job seekers with a disability were above average. This is expected given their greater level of disadvantage compared to all Job Network eligible job seekers.
Young people appeared more satisfied than other job seekers with Centrelink services, although qualitative research reveals that young job seekers are more confused about the services and obligations associated with Job Network and are less likely to complain than older job seekers. Young people were more critical than other job seekers when surveyed about the level of information Job Network members provided on their rights and responsibilities and relevant rules and regulations. The rate of assessment and the rate at which youth are selected for Intensive Assistance are below that for other job seekers. These trends may partly be explained by the fact that participation is voluntary for young people not on income support.
Older workers’ satisfaction levels with employment services were similar to those of other job seekers, although many thought that they could be better marketed by Job Network members to employers who they perceive to be reluctant to take on job seekers older than 45. In qualitative research, some older workers expressed concern about the appropriateness and content of some Job Search Training courses.
Compared to other job seekers, indigenous Australians were less likely to have been assessed. Those who have been assessed had a high rate of selection for Intensive Assistance, but many referred to Intensive Assistance did not commence. For indigenous Australians who participate in Intensive Assistance, concerns have been raised in qualitative interviews with Job Network members, Centrelink staff and indigenous job seekers about the quality and type of assistance delivered.
The Government has responded to concerns about indigenous job seekers’ access to employment services by introducing a number of reforms, including the Indigenous Employment Policy (in July 1999), changes to Job Network in the second tender round and new initiatives for Centrelink.
Progress on implementing the reform principles
While this report has focused on the implementation of Job Network and the experiences of stakeholders, it has also started consideration of some of the broader issues about building a competitive market. The principles that underpin the reforms include integration of income support and labour market assistance, targeting job seekers at risk of remaining unemployed (discussed above), flexibility in service delivery, contestability through competitive tendering, local choice-driven competition between multiple providers, and performance-based funding that rewards desirable outcomes.
Establishment of Centrelink achieved integration of income support and access to labour market assistance; the Government did not pursue integration of the delivery of employment services. The evaluation found that some job seekers and some employers did not fully understand how Job Network operates or the full range of the services available. However, this needs to be viewed in light of the magnitude of the changes undertaken and the early stage of market development.
Eligibility restrictions ensure limited resources are targeted to job seekers most in need. Devolution is a key element of allowing providers the flexibility appropriate to meeting job seeker needs. However, because providers make judgments about how much assistance to provide this can affect equity and quality of service to job seekers.
Flexibility also means that Job Network members can pursue innovative ways of assisting job seekers, particularly in Intensive Assistance. Flexibility, however, needs to be balanced against ensuring sufficient accountability for expenditure and equity of servicing, particularly for hard-to-place job seekers. This balance is pursued in Job Network through a range of contractual provisions, quality assurance processes, rigorous performance monitoring arrangements and a complaints mechanism.
The open tendering process resulted in a substantial increase in the number of sites and range of providers of employment services compared to previous arrangements. Employers and job seekers can make their own choices about providers. This practice will improve over time, with the release of performance information and increasing familiarity with the process of choosing.
Use of outcome payments provides incentives to Job Network members in two ways. First, they provide a financial reward and, second, they are a clear measure enabling comparison with other providers. This is important given the weight placed on performance in the second tender round. Equity and value for money can be a concern in Intensive Assistance should the fee structure not provide sufficient incentive to help the most disadvantaged. Further analysis is planned to see if changes to the fee structure in the second tender round facilitate greater equity in service provision.
Areas for further research
Further research is warranted on a number of broader policy issues not covered in detail in this report. This includes examining:
- How well the current mechanisms for maintaining service quality in a competitive environment are working. These mechanisms include service standards, use of information on past performance in assessing tenders, the provider code of conduct, complaints mechanisms, quality auditing, and the feedback and performance improvement processes.
- How competition and devolution impact on quality and effectiveness. For example, while providing a strong incentive for improvements in service, competition may restrict the amount of information sharing between providers. Furthermore, large and small providers may have different approaches to information sharing based on their capacity to research the most effective forms of assistance for job seekers.
- The role of Government in developing and sharing information. Given the flexibility in Job Network for providers to trial different approaches to assisting job seekers, how can the Government distribute this information to all Job Network members while supporting a competitive environment?
- Whether there are effective alternatives to the full open tender process, such as extension of contracts for high performing providers.
- The operation of Job Network at the local labour market level in order to measure regional variations in access, responsiveness and market development.
There are other aspects of the implementation of Job Network that may require further examination. These include the development of relationships between stakeholders involved in the effective functioning of an integrated employment assistance and income support system, and the impact of performance information on job seeker choice during the second contract period. In addition, the Department of Employment and Workplace Relations is currently doing further research into the type and volume of assistance provided to job seekers in Intensive Assistance.
The published report is available on request from
- Evaluation & Program Performance Branch
Dept of Employment and Workplace Relations
GPO Box 9879
Canberra ACT 2601
- fax: 02 6121 6027
- Email
February 2000