United States Employment Service; labor exchange performance measures,

[Federal Register: June 12, 1998 (Volume 63, Number 113)]

[Notices]

[Page 32563-32578]

From the Federal Register Online via GPO Access [wais.access.gpo.gov]

[DOCID:fr12jn98-161]

[[Page 32563]]

Part V

Department of Labor

Employment and Training Administration

United States Employment Service; Labor Exchange Performance Measures; Notice

[[Page 32564]]

DEPARTMENT OF LABOR

Employment and Training Administration

United States Employment Service; Labor Exchange Performance Measures

AGENCY: Employment and Training Administration.

ACTION: Notice and request for comments.

SUMMARY: In response to the requirements of the Government Performance and Results Act (GPRA) of 1993, the national call for government programs to be more accountable and results-oriented, the Department of Labor (DOL) Employment and Training Administration (ETA) launched a project called the Workforce Development Performance Measures Initiative (WDPMI) to develop a menu of key performance measures for use in the workforce development system. Due to the absence of established key performance indicators for the public labor exchange program that measures both the self-service and staff-assisted service options now available, the United States Employment Service (USES) initiated a project at about the same time to work cooperatively with States in developing program-specific performance measures. This Federal-State workgroup also was charged with ensuring that the labor exchange performance measures it proposed would be compatible with and complementary to the overall WDPMI performance measures being developed.

The Labor Exchange Performance Measures workgroup prepared a discussion draft issues paper entitled ``America's Labor Exchange Performance Measures'' that identified a list of potential measures. This paper was shared informally among the State Employment Security Agencies during a period beginning on September 10, 1997, until October 31, 1997. More than 20 States provided reactions and feedback on the proposed measures. The Labor Exchange Performance Measures workgroup considered these comments and in subsequent discussions refined the potential program measures.

This Federal Register Notice (FRN) provides a description of the conceptual framework within which public labor exchange services are delivered, and requests comment from interested parties and USES' stakeholders on proposed performance measures for labor exchange services. The Department of Labor also is interested in comments on the appropriate number of measures, and whether the proposed measures take into account the full range of services and service options in the modern labor exchange. In addition, the Department is interested in learning about other measures that State labor exchange agencies have found useful for management and continuous improvement purposes.

There currently are two bills being considered by the Congress (H.R. 1385 and S. 1186) which will provide the framework for the Nation's workforce development system. Among the provisions of the bills is a requirement that the Wagner-Peyser funded labor exchange program functions are provided through the One-Stop system (called ``full-service'' in H.R. 1385).

These proposed measures are a starting point for development of comprehensive measures for the labor exchange function of the emerging workforce development system. This FRN does not address the data elements needed to produce the performance measures nor proposes specific changes to the ETA reporting requirements. That will be the subject of a subsequent notice.

DATES: Comments on the proposed Labor Exchange function performance measures must be received by the U.S. Department of Labor on or before July 27, 1998. Late-filedcomments will be considered to the extent possible.

ADDRESSES: Comments must be filedin Room N4470, U.S. Department of Labor, Employment and Training Administration, United States Employment Service, 200 Constitution Avenue NW., Washington, DC 20210.

FOR FURTHER INFORMATION CONTACT: John R. Beverly, III, United States Employment Service, U.S. Department of Labor, 200 Constitution Avenue NW., Room N4470, Washington, DC 20210, Tel. 202-219-5257, Fax 202-219- 6643, E-mail jbeverly@doleta.gov

SUPPLEMENTARY INFORMATION:

Contents

1. Authority

2. Introduction

   1. Mission of the US Department of Labor B. Role of the Public Labor Exchange

3. Performance Measures Development Process

   1. Labor Exchange Performance Measures Workgroup B. Models for Service in the Modern Labor Exchange

   1. Self-Service strategy

   2. Facilitated Self-Help strategy

   3. Staff-Assisted strategy C. Labor Exchange Performance Measurement Strategy D. Initial Comments on the Draft for Discussion Issues Paper

4. Definitions

   1. Job Search Assistance B. Facilitated Self-help C. Self-help Tools D. Staff Assisted Labor Exchange Services E. Job Listing Services F. Referral Related Action G. Business Assistance Services H. Entered Employment I. Entered Employment Rate J. Cost per Entered Employment

5. Proposed Labor Exchange Performance Measures

   1. For Self-Service Strategies

   1. Holdings

   2. Users

   3. Transactions B. For Facilitated Self-help Strategies

   1. Number of Job Seekers Using Self-Help Service

   2. Customer Satisfaction C. For Staff-assisted Service Strategies

      Measures of Job Seeker Customer Services

   1. Entered Employment Rate

      [Option] Job Development Entered Employment Rate

      Measures of Employer Customer Services

   2. Job Listing Return Business Rate

   3. Business Assistance Service Return Business

   4. Referral Response Time

   5. Average Time Lapse to Successful Referral

   6. Job Order Fill Rate D. System Measures

   1. Cost Per Entered Employment

   2. Duration of Benefits Compensated

1. Authority

   Labor exchange performance measures will be implemented under the following authority:

   1. Wagner-Peyser Act section 10 (c), 29 U.S.C. 49i(c)

      (c) Reports

      Each State receiving funds under this Act shall (1) make such reports concerning its operations and expenditures in such form and containing such information as shall be prescribed by the Secretary, and (2) establish and maintain a management system in accordance with guidelines established by the Secretary designed to facilitate the compilation and analysis of programmatic and financial data necessary for reporting, monitoring and evaluating purposes.

   2. Government Performance and Results Act of 1993

      Purposes--The purposes of this Act are to improve the confidence of the American people in the capability of the Federal Government, by systematically holding Federal agencies accountable for achieving program results; --improve Federal program effectiveness and public accountability by promoting a new focus on results,

      [[Page 32565]]

      service quality, and customer satisfaction.

2. Introduction

   1. Mission of the US Department of Labor

      The 1998 USDOL Strategic Performance Plan states that it is the mission of the Department of Labor to foster and promote the welfare of the job seekers, wage earners and retirees of the United States by improving their working conditions, advancing their opportunities for profitable employment, and protecting their retirement, health, and other benefits.

      In carrying out a part of this mission, the Department, through the Employment and Training Administration, administers a variety of Federal programs to help employers find workers, to help job seekers find employment, to provide unemployment insurance benefits and reemployment services to jobless workers, to track changes in employment and to provide job training to meet the changing workplace skill needs.

   2. Role of the Public Labor Exchange in the USDOL Mission

      In a May 1, 1933, report to the Senate from the Education and Labor Committee, and a May 22, 1933, report to the House of Representatives from the Labor Committee, the stated purpose of the public labor exchange is:

      (1) To foster, promote and develop the welfare of the wage earners of the United States including juniors (youth) regularly employed;

      (2) To improve their working conditions;

      (3) To advance their opportunities for profitable employment by regularly collecting, furnishing, and publishing employment information as to opportunities for employment;

      (4) For maintaining a system for clearing labor between the several States; and

      (5) For cooperating and coordinating the public employment offices throughout the country.

      On June 6, 1933, the Wagner-Peyser Act (29 U.S.C. 49 et seq.) became law and has since provided the basic funding support for the public labor exchange system.

      Over the 64 years of its history, the role of the public labor exchange has changed and evolved to meet the needs of the nation, employers and job seekers. Dedicated and hard working employees have labored in service to the United States from its earliest role in assisting the nation to recover from the depths of the Great Depression as part of the national recovery effort, through the efforts in support of the nation's economy by assisting private sector employers as their need for workers was revived. In the years that followed, the public labor exchange was an important part of the mobilization in support of war industries during World War II and the return to a post war economy. During that time the federal/State partnership was restructured with a redirection of effort to support private sector employers' workforce needs. Through the 1960's and 70's the national system of Wagner-Peyser funded labor exchange offices was instrumental in advancing a variety of social programs designed to expand participation and conditions of employment for several target groups including jobless workers, the economically disadvantaged and those who had particular difficulty in gaining access to employment opportunities.

      In 1986, the Secretary of Labor conducted a series of four public hearings at which 138 individuals testified and 562 submitted written testimony representing a cross section of the stakeholders of the nation's public labor exchange, including business and industry, the unemployed, target group advocates (e.g. veterans etc.), labor unions, State governments, employees and the training and education communities. The intent was to assess the current role of the labor exchange and to secure input on the appropriate role for the Wagner- Peyser Act funded labor exchange in the 21st century.

      The vast majority of respondents (97.6%) indicated that the nationwide public labor exchange system was viable, and should be maintained and improved. Many of the respondents highlighted the need to streamline the system and to return to providing basic labor exchange activities. The list which follows represents the most frequently identified services in their order of priority:

      ‹bullet› Intake, general assessment, and referral to jobs, training or re-training opportunities, and/ or support services;

      ‹bullet› Labor market and occupational information;

      ‹bullet› Preliminary screening of applicants for employers' job orders;

      ‹bullet› Basic skill and/or aptitude testing;

      ‹bullet› Direct placement (includes job referral);

      ‹bullet› Job Search Assistance, including training and tips on interviewing, etc.; and

      ‹bullet› Specialized recruitment for employers with large staffing requirements and workforce needs.

      The majority of commentators indicated that the public labor exchange services should be available free to all employers and job seekers, and that the role of the federal, State and local governments and the private sector should be maintained as they were at the time of the review and are today.

      In addition to responding to specific questions posed in the FRN that announced the hearings, the commentators also offered other recommendations including:

      ‹bullet› The public image of the labor exchange should be improved to promote use of the available services,

      ‹bullet› There be an increased use of automation and technology to provide linkage among states and localities and related programs to assist job matching, and to provide more timely and accurate labor force projections,

      ‹bullet› DOL develop better standards to evaluate performance and incentive systems,

      ‹bullet› Staffing levels be increased,

      ‹bullet› Data collection and reporting systems be enhanced,

      ‹bullet› Staff training be increased,

      ‹bullet› Better job skills assessment tools be developed,

      ‹bullet› Labor market information be improved,

      ‹bullet› Federal support for State-level program integration efforts be increased, and

      ‹bullet› Alternative funding sources to finance operations be found.

      In the 12 years since the re-examination, the public labor exchange system has been streamlined. Revitalized operations now provide service using flexible tiered service delivery strategies. The public labor exchange has significantly increased the use of automation and technology to provide service and information, and implemented many of the changes and improvements which were suggested. In cooperation with the Unemployment Insurance Service, the public labor exchange also developed a system of worker profiling and re-employment services for Unemployment Insurance (UI) claimants to speed their return to productive employment.

      The Wagner-Peyser funded public labor exchange has become a core component of the new Workforce Development System and is the universal access component of the nation's One-Stop Career Centers. For many business and job seeking customers, their only experience and contact with the Workforce Development System will be Wagner-Peyser funded services.

      The lack of additional financial investment in the public labor exchange may be traced, to some degree, to its failure to develop national labor

      [[Page 32566]]

      exchange performance measures. Performance data that have been collected and used provides an inaccurate picture of the overall success and value of the services provided. Moreover, these data fail to address the employer customer who is taxed to pay for the public labor exchange services.

      In this fiscal environment where labor exchange funding has either remained the same or declined, a good performance measurement system will help States and agencies manage programs and decide on the most appropriate mix of labor exchange services to offer.

      For the most part, previous measurement schemes were simple counts of service outputs, e.g., individuals referred to jobs, with the effectiveness of those outputs gauged through ratios made by comparing those counts to the total pool of persons registering with the labor exchange, e.g., 24% of all labor exchange applicants received a job referral. Moreover, measurement of the effectiveness of employer services were confined to a count of the number of job openings received by the public labor exchange as compared to the number of openings on which a placement action occurred, i.e., the job openings filled rate.

      The ETA 9002 reporting and data collection system has been the source of these program data. The authority to collect ETA 9002 data will expire on August 31, 1999. This sunset date offers an opportunity to re-examine how the labor exchange function is measured. It is DOL's hope to secure comment from all of the stakeholders including employers, wage earners, unions, and partner agencies in the emerging workforce development system, including the education, training, and public assistance communities on both the performance measures proposed herein and the conceptual framework within which those measures would operate. The comments received will be reviewed and adjustments will be made to the proposed measures as appropriate. Following this participatory process, another FRN will be issued to promulgate the key performance measures, to identify the data elements needed to produce those measures, and to establish State data reporting requirements.

      ETA recognizes its responsibility to continue to collect information on the race, age, sex, religion, ethnicity, disability status, and veteran status of the job seekers who register to use its services.

3. Performance Measurement Development Process

   1. Labor Exchange Performance Measures Workgroup

      The USES Labor Exchange Performance Measures Initiative (LEPMI) Workgroup was composed of fifteen representatives drawn from State employment service agencies, USDOL regional and national offices, the Veterans' Employment and Training Service, and the Interstate Conference of Employment Security Agencies (ICESA). The members of the Labor Exchange Performance Measures Initiative (LEPMI) possessed on average over 20 years of experience in a variety of program areas and disciplines including tracking and monitoring program performance.

      The workgroup was convened in June 1997 to develop a menu of conceptual performance measures which could be adopted for the Labor Exchange function of the Workforce Development System. The Workgroup used guidelines similar to those used by the ETA Workforce Development System Performance Measures Initiative Policy Committee to assess alternative measures, and maintained a close coordination between that effort and its work. Several persons served on both workgroups. This close working relationship between the two complementary performance measures initiatives has continued.

      The Workgroup began by agreeing that the new performance measurement system should be directed toward measuring the outcomes of service delivery, whenever possible, and not the inputs or outputs of the processes used to provide service. The first task the LEPMI work group tackled was to identify the core services being provided in today's labor exchange. The workgroup identified the following as the core service categories of the public labor exchange: for Employers the core services identified were Job Listing, Business Assistance, and Job Matching and Initial Screening of Candidates for Employment; for Job Seekers the core services identified were Job Search Assistance, Job Development, and Facilitating Access to Self-Help Tools.

      There are a wide variety of activities which are included within the broad categories of service, examples of which can be seen in Figure #1 which follows this FRN.

      To develop performance measures, the workgroup benchmarked a variety of measurement strategies including the way in which the results of the public labor exchange has been measured in the past, workload-based measures similar to measures used in UI programs, measures of the effectiveness of training programs, approaches used to measure private sector business success and indicators of success in the public library system.

      The workgroup also agreed early in its deliberations to a set of four guiding principles:

      1. The Performance Measures proposed are intended to apply to the labor exchange function without regard to the administrative or program structure in which labor exchange services are delivered. Traditionally, the Labor Exchange function has been within the purview of State Employment Security Agencies (SESA's) nationwide. With the emergence of the Electronic Labor Exchange, worker profiling and intensive job search assistance activities under the Dislocated Worker programs, Welfare Reform, and One-Stop Career Center service delivery approaches, the Labor Exchange function is now being performed in a variety of agency and program models. These measures focus on the function not the program.

      2. The customer universe against which staff-assisted service outcomes are to be measured would be limited to those persons who have received staff-assisted services and not all persons registered with the public labor exchange, whether or not they are seeking work.

         There are several customer groups who use the public labor exchange for purposes other than immediate employment. These include but are not limited to applicants and recipients of Food Stamps, Home Relief or State General Assistance, Medicaid, temporary housing, and day care programs. Many of these programs require registration with the public employment service as part of the program application process.

         Other customers have an intermittent interest in pursuing employment. These can include applicants who are not in the labor market and those who are currently employed but interested in new job opportunities. Many of these individuals will periodically register for employment with the labor exchange. A segment of this group includes candidates for employment with prominent local firms that have developed exclusive recruitment agreements with the public labor exchange office. Some employers, such as auto industry manufacturing companies and major retail outlets, post signs outside of their plants and personnel offices which inform candidates that they hire candidates for employment who are referred by the

         [[Page 32567]]

         public labor exchange only, directing the job seekers to complete applications at a local labor exchange office.

         Mass recruitment control services are valued by many employers and result in registration of thousands of job seekers who are interested in working for only one of these major area employers. If any of the members of these customer groups fail to respond to other employer recruitments or job referral call-ins, and never return to the offices for job search assistance, they will ultimately be inactivated with no service. These customers are excluded from consideration when calculating the outcomes for labor exchange service because they did not receive and were not interested in staff-assisted service.

      3. Although the performance measures presented in this FRN are not segmented by target group, e.g., Welfare-to-Work, Migrant and Seasonal Farm Workers, etc., ETA encourages segmenting performance measures to allow a comparison among groups of customers. Measuring program performance based simply on overall entered employment outcomes, without taking into account the labor market characteristics of the various target groups of job seekers being serviced, may under- represent the true performance of the public labor exchange system. Because public labor exchange services are available to all job seekers and all employers (universal access), its performance outcomes routinely include hard-to-place customer groups, e.g., potential UI Exhaustees, Public Assistance Recipients, Migrant/Seasonal Farm Workers, or Youth. However, current methodology does not account for the difficulty factors these groups may present. Segmentation of performance measures may offer insight into these overall outcomes. For example, an entered employment rate might be developed for Welfare-to- Work job seekers which could be compared to the entered employment rate for all job seekers.

         ETA believes segmentation of outcomes for specific groups of customers can provide valuable insights into overall performance outcomes. This FRN does not address the data elements needed to produce the segmented outcome measures nor does it propose specific changes to ETA reporting. These will be the subject of a subsequent notice.

      4. Data gathering, to the extent possible, should be directly related to service delivery and the cost of data collection and reporting should be proportionate to the cost of the service provided. A performance measurement system developed in the current environment that has been shaped by the national effort to reduce the administrative paperwork burden and to focus on the results of government programs should not propose measures that cost substantially more to collect and report than the unit price for delivering the service.

   2. Service Delivery Strategies in the Modern Labor Exchange

      To continue to meet customer needs for labor exchange services in a period of diminishing resources, States operating Employment Security, One-Stop, and Workforce Development programs have devised a variety of approaches and service delivery models which are significantly different from the traditional model of an ``Unemployment'' office that provided one-on-one staff-intensive interviews and individualized assistance to job seekers in finding jobs. This model, as the sole means delivery of labor exchange, is now largely extinct.

      Today, labor exchange services are typically provided using a tiered delivery system composed of three flexible and adaptive service strategies: ``Self-Service''; ``Facilitated Self-Help''; and ``Staff- Assisted Service.'' Figure #2, which follows this FRN, shows this service model. Each strategy is designed to respond to differing service needs and differing service populations. It is the Department's expectation that State agencies providing labor exchange service will use each of the three service delivery strategies.

      This FRN represents the first effort to describe the tiered service delivery continuum, particularly the electronic Self-Service and Facilitated Self-Help strategies. Accordingly, more descriptive text has been devoted to the descriptions of the Self-Service and Facilitated Self-Help service strategies compared to the description of the traditional Staff-Assisted service strategy. This should not be interpreted as a preference for one strategy over another. Each has a necessary and appropriate role in labor exchange service delivery.

      1. In a Self-Service strategy, States make labor exchange resources available which customers can utilize independently, i.e., without staff intervention. Three current trends have helped drive the development of self-service strategies in today's public labor exchange.

         First, government is increasingly required to do more with less, and so must find ways to deliver its products and services more efficiently.

         Second, the labor market itself has changed, with unemployment being less associated with cyclical pressures and more with structural causes where increasing numbers of job seekers are considered dislocated workers. Studies have suggested that in the future job market most workers will change careers (not just jobs) several times during their work lives.

         Third, customer expectations have changed, and the public labor exchange system faces new demands from its customers regarding the quality of the services that are provided. Customers also expect to participate in the decisions involving what services are provided and how they are provided.

         This delivery mode places the customers in charge of the services they receive. The acceptance of self-service modes of service delivery is demonstrated by the increased number and availability of public access computers, automated teller machines, debit cards, and library swipe cards with bar-coded data, etc. There has also been a general movement within the United States toward placing more responsibility on, and expecting greater participation from, an individual who wishes to receive governmental services or benefits. In line with these trends, customers of the public labor exchange increasingly have made clear their preference for exercising informed choices in determining which products and services they receive and how they receive them.

         An additional and significant benefit of the self-service mode of service delivery is that self-service expands the capacity of the system beyond the limits of the available staff resources, thereby efficiently handling a wide variety of customer labor market and employment information needs.

      2. In the Facilitated Self-Help strategy, customers are provided access to self-help resources at the One-Stop Career Center or local labor exchange office, generally through a dedicated Resource Room. For many customers this is their first exposure to self-service tools and computer-based systems. In addition, one or more staff are assigned to assist customers who need help in using those resources. These staff interact with the customer, as needed, to facilitate the customer's job search using government-provided resources, e.g., personal computers, word processing and/or resume writing software, fax and copy machines, etc., and online access to the DOL-funded Internet-based tools, e.g., America's Job Bank (AJB), America's Talent Bank (ATB), America's Career InfoNet, etc. After being introduced to and assisted to use these Internet-based

         [[Page 32568]]

         self-service systems many customers will be able to use them in the future without assistance both in the One-Stop Center and in other locations with access to the Internet, e.g., schools, libraries, and at home. As funding for the traditional labor exchange or employment service has remained flat or declined, use of this mode of service delivery has increased. It continues to place the customers in charge of the services they wish to access, but provides the staff facilitation needed to ensure that the customers can obtain the service or information they require. Again, the cost for providing services using this strategy is less than under a Staff-Assisted service strategy, since the ratio of staff to customers is much higher.

      3. The Staff-Assisted strategy ranges from intensive one-on-one services where a staff person is assigned to a job seeker as a case manager, to those where a customer interacts with service staff in a small group setting. This is the more traditional mode of operation for delivery of public labor exchange services. The services most frequently delivered in this manner are assessment, intensive job search assistance, employment plan development, case management, counseling and vocational guidance, and job development. The expected output is a job referral and the expected outcome is entry to employment. Group services, such as job clubs and workshops on interviewing techniques, the world of work, and use of labor market information are also classified as staff-assisted services, although they are provided on a one-to-many service basis.

   3. Discussion of Labor Exchange Performance Measurement Approaches

      DOL is proposing a different approach for measuring performance under each of these service strategies. The Department does this for two primary reasons. The first reason is that each strategy results in significantly different relative costs. Costs can be measured per unit of service provided or as an overall cost for providing access to resources. The costs to provide a service should be the primary determinant in deciding how much to invest to measure that service and how much customer information to collect. The more significant the service intervention, e.g., the higher the unit cost of that service, the greater the rationale for collecting data to justify the expenditure of public funds. For example it would not appear to be prudent to spend $10.00 pursuing a follow-up action to determine the outcome of a service that costs less than 25 cents to deliver. However, one should be willing to expend significant resources in determining whether a $5000 investment in training results in a long term payoff to the taxpayer.

      The following are examples of labor exchange services and resources that might routinely be provided in a self-service or facilitated self- help mode and for which it might be difficult to document an employment outcome without a substantial data collection and follow up effort.

      ‹bullet› Labor Market Information.

      ‹bullet› Self-help pamphlets, magazines, newspapers, and reference books.

      ‹bullet› Internet access for use of self-help job finding tools, e.g., AJB/ATB.

      ‹bullet› Access to computer hardware and software, fax and copy machines, telephones, and other equipment in resource centers.

      Gaining Internet access to resources and tools such as these from a job seeker's home or from a community resource, such as a library or community center, may cost the public labor exchange system only a few pennies per transaction. First, because economies of scale are achieved in the development and maintenance of Internet-based systems and resources. Second, because the cost of the computer equipment, Internet service provider access, telecommunications software and hardware, supplies, electric power, and facilities are borne by the user or by another organization.

      This cost per transaction rises slightly when these latter costs are borne directly by the public employment service, such as when offered through a resource room. When these self-service resources are further augmented by making staff resources available to assist and/or instruct customers in using these tools and resources, the unit cost again rises.

      A similar case might be made for employer services. An employer may post an unsuppressed (broadcast) job opening directly to AJB over the Internet instructing interested parties to contact that employer directly. This is a very low cost self-service option.

      Employers may also request assistance from public labor exchange staff in placing their job openings on America's Job Bank. The cost of this service can increase further when the employer also asks the Labor Exchange to screen and refer applicants to that job order. The cost to provide this level of staff-assisted service may rise to the level where tracking outcome data is important.

      The second reason for pursuing different kinds of performance measures when using different service delivery strategies is that, for program outcome measures, one must be able to make a rational argument that there is a nexus between the service(s) provided and the outcome measured. For example, it might be difficult to argue that a single user session of browsing through labor market information in a self- service mode over the Internet can be directly linked to a person obtaining a job 30 days later. However, should that same person come into the local public labor exchange office several times over the course of a few weeks and attend workshops, obtain help in his or her job search from the local office staff, and receive one or more job referrals, such a nexus can be reasonably concluded.

      Clearly, finding a way to gauge the results of self-service and facilitated self-help strategies presents a challenge to the system. This is particularly true since the Government Performance and Results Act (GPRA) became law. Under GPRA, government agencies must account to their investors (Congress and the American people) as to both how their programs perform and what results they achieved. The challenge is to find a means to satisfy this requirement where the service costs only a few pennies per transaction to deliver and the relationship to traditional program results is difficult to draw. As the level of service intervention increases, staff involvement grows, and dollar investment rises, the use of outcome and impact performance measures may become necessary to justify and explain the expenditure of public resources.

      DOL hopes that it has addressed these concerns in the performance measures which follow. 1. Measurement Approach for Self-Service Strategies

      A small ad-hoc task team was convened to come up with an approach to measuring the self-service components of the workforce development system. This team included several persons who were members of both the Workforce Development Performance Measure Initiative (WDPMI) and the Labor Exchange Performance Measures Initiative (LEPMI) workgroups. The team concluded that the following factors should be used to determine when outcomes measures would be appropriate for gauging the performance of a particular self-service or facilitated self-help service:

      ‹bullet› There is a significant value-added provided by the service, i.e., the service is more than just an information exchange.

      [[Page 32569]]

      ‹bullet› The cost to collect and track outcomes is less than the cost per unit to provide the service.

      ‹bullet› The service itself is intended to lead to an employment or other measurable outcome.

      ‹bullet› The cost of providing the service rises to a point which indicates a need to justify the expenditure of public resources.

      ‹bullet› The time lapse between receiving the service and the measurement of the outcome is short enough that the service can be considered to be causative of and/or contributory to that outcome.

      The Ad-hoc Task Team used these factors to help determine whether self-service and facilitated self-help strategies should be included in outcome measurement calculations. The first group of services considered were those which could be provided in a completely self- service mode, requiring no staff assistance. These services included:

      ‹bullet› accessing and using electronic labor exchange information services,

      ‹bullet› completing online self-assessments,

      ‹bullet› finding information, such as labor market information (LMI) or community services,

      ‹bullet› online training, self improvement, and skill enhancement, and

      ‹bullet› applying for services and benefits (e.g., using telephones for original and continued UI claims).

      The Ad-hoc Task Team agreed that customers receiving services in this fashion should not be included in outcome measurements, but rather services should be measured using output measures.

      In general, the Ad-hoc Task Team agreed with the performance measurement strategy which had been developed by the Labor Exchange Performance Measures workgroup. Under this strategy, measuring the value of self-service strategies, such as using DOL and State-developed Internet-based tools (America's Job Bank, America's Talent Bank, and America's Career InfoNet) should be analogous to measuring the value of services measured in public library systems. Using this concept, performance would be measured by counting the growth over time of:

      ‹bullet› Holdings: the numbers of jobs and resumes that are maintained in its inventory;

      ‹bullet› Usage: the number of times customers use its services; and

      ‹bullet› Transactions: how many times specific kinds of services are used, e.g., referral information on job listings are accessed or downloads of resumes are requested.

      Figure #3, which follows this FRN, shows a model of this approach to measure program performance. 2. Measurement Approach for Facilitated Self-Help Strategies

      Under the facilitated self-help service strategy, staff assist customers to use self-help tools and resources. The services provided include:

      ‹bullet› the self-service activities noted above;

      ‹bullet› access to computer hardware, software, telephones, office equipment and other physical resources in a Labor Exchange office; and

      ‹bullet› staff assistance in using any of the above services or resources.

      The approach to measurement of these services would be a combination of usage and customers' satisfaction. Usage would be measured as a simple count of the number of persons using resources rooms. This could be collected using swipe cards, tallies, sign-in sheets, and/or automated session counts. Customers using self-help services should not be included in outcome measurement until or unless they are provided more intensive staff-assisted service. Customer satisfaction would be used to look at the dimensions of service: accessibility, quality, timeliness, and security. This could be done by surveys that would not be reported nationally. These surveys would be locally developed, controlled, and used for continuous improvement projects at the local level. 3. Measurement Approach for Staff-Assisted Strategies

      Services delivered using a staff-assisted service strategy represent a value-added service. Measuring the performance of these value-added services requires addressing their employment-related outcomes for job seekers and employers. Additionally, measures should be adopted which can be used to assess the overall effectiveness and impact of the public labor exchange system.

   4. Initial Comments on the Draft for Discussion Issues Paper

      The United States Employment Service in cooperation with the Interstate Conference of Employment Security Agencies (ICESA) released the issues paper, entitled, ``Draft for Discussion--America's Labor Exchange Performance Measures,'' in September, 1997. It provided a review of the rationale and a suggested conceptual framework for establishing a set of key performance measures of labor exchange services. Included was a discussion of a variety of alternative measures and the identification of 10 potential performance measures of staff-assisted job seeker and employer services. In addition, performance measures were suggested for the electronic labor exchange which counts holdings, usage, and transactions. Also included were two system measures. One was a cost-to-benefit measure that looks at unit cost per outcome, and the other was a measure of the impact of service on Unemployment Insurance benefits compensated.

      State Employment Security Agency Administrators were asked to provide written reactions to the suggested measures by October 31, 1997. The ``Draft for Discussion'' issue paper was also made available on the Internet on the USDOL-supported ICESA Workforce ATM website (http://www.icesa.org/national/docs/LABOREX.HTML). This approach provided an opportunity for the public labor exchange system to have direct input to the initial development of the performance measures that are the subject of this FRN.

      Based on the comments received from States, and subsequent additional discussion and consideration by the workgroup, the number of job seeker measures has been reduced and several improvements have been made to other measures. In some cases, where different approaches were suggested and there was no clear-cut decision as to which approach was the better choice, more than one option has been included in this FRN for comment.

      Current system redesign efforts to enhance and integrate the features of America's Job Bank and America's Talent Bank systems will make it possible to collect and aggregate some data electronically without additional State data collection or transmission effort. It is the Department's intention to actively pursue electronic data collection approaches to the extent feasible.

4. Definitions

   1. Job Search Assistance (JSA): Includes services currently defined in the ETA 9002 Data Preparation Handbook NO. 406 under the ``Received Some Reportable Service'' categories and in items 12 through 42. The service activities included are: job search workshops, job finding clubs, resume assistance, providing specific labor market information, job search plan development, job matching, job development, referral to jobs, vocational guidance, assessment interviews, testing, vocational counseling, federal bonding, and referral to other services

      [[Page 32570]]

      (including skills training, educational services and supportive services).

   2. Facilitated Self-help: Is a staff-assisted service in which staff facilitates customer access to a variety of in-office self-help job finding tools and resources. Support staff provides limited assistance on an as needed basis. Most job seekers require limited assistance. Normally, job-seeking customers are not required to register for service before they are allowed access to self-help tools or resource areas, rooms or centers.

   3. Self-help Tools: Include, but are not limited to, resource rooms, automated labor exchange system devices, Internet access, fax machines, telephones, photocopiers, personal computers, word processors, career and labor market information, and reference materials.

   4. Staff Assisted Labor Exchange Services: Include job search assistance, job development assistance and job referrals, and can be characterized as service which is conducted one-on-one or in small groups.

   5. Job Listing Services (JLS): Include activities performed on behalf of employers to assist them in filling their workforce needs. JLS includes but is not limited to services and activities such as job order taking, job order maintenance, referral follow-up and interview support, together with connecting activities including searching job seeker databases, transmitting resumes, and marketing job orders to the job seeker (applicant) pool, etc.

   6. Referral Related Action: Includes the following services and activities:

      (1) Referral of qualified candidates, or

      (2) Contact with the employer to review potential referrals, to develop a candidate recruitment strategy or adjust features of the job order which prevent identification of candidates for referral, i.e., salary, experience requirements, etc.

   7. Business Assistance Service (BAS): Includes, but is not limited to, providing employer education seminars, job and task analysis, providing local labor market information, and referral to other workforce development or economic development services or agencies.

      There are a number of employer service activities which are excluded from the definition of BAS. They are: periodic mass mailings and routine promotional mailings to increase employer job listings, providing labor market information via the Internet and employer contacts for the provision of other core services, e.g., job listing, job matching, initial screening or referral services.

   8. Entered Employment: The unduplicated count of job seekers (applicants) who enter employment by job placement or obtained employment. See page II-12 of ETA 9002 Data Preparation Handbook, ET Handbook NO. 406.

      (1) Job Placement: The hiring of a job seeker by a private or public employer after referral to a job by the Agency staff, or collocated or out-stationed staff in cooperation with the Agency, provided that the following conditions were fulfilled:

      (a) Prepared a job order prior to referral, except in the case of a job development contact on behalf of a specific job seeker,

      (b) Made prior referral arrangement with the employer,

      (c) Referred an individual who was not designated by the employer, and

      (d) Verified from a reliable source that the job seeker had entered work, and the placement was recorded in the agency data base.

      See page II-12 of ETA 9002 Data Preparation Handbook ET Handbook NO. 406.

      (2) Obtained Employment: Individuals who secure employment within the current quarter or the next completed quarter following the last staff-assisted job search assistance service that was partially funded by the Agency, such as:

      (a) Participating in job search activities,

      (b) Accepting a position resulting from the use of an agency- sponsored automated labor exchange,

      (c) Receiving employment counseling or testing or development of an employability plan,

      (d) Receiving bonding assistance,

      (e) Terminating from a skill training program to which a job seeker was referred by the Agency, or

      (f) Receiving tax credit voucher, and receipt of verification from a reliable source, preferably the employer.

      See page II-13 of ETA 9002 Data Preparation Handbook ETA NO. 406. Entered Employment can be counted in the current quarter or the next completed quarter following the receipt of the last staff-assisted service (a maximum of 180 days).

1. Entered Employment Rate (EER): Is the percentage of job-seekers securing employment after receiving Staff-Assisted Services.

   10. Cost per Entered Employment (CPEE): Is the cost of achieving the positive outcome of entry to employment following the provision of service. The CPEE is determined by dividing the total of those who entered employment by the total funding received for the federal Wagner-Peyser Act, and Veteran Services grants and State appropriations used for the Labor Exchange function during a program year (July 1-June 30).

5. Proposed Labor Exchange Performance Measures

   Specific questions have been developed regarding these performance measures for which public comment is sought. Questions can be found at the end of sections A., C., and D. below, and they are numbered sequentially, 1 through 16. Commentators need not repeat questions as part of their response, but addressing the specific questions by number would be most helpful to this effort.

   1. For Self-Service Strategies

      Performance measures will be output measures based on holdings, usage and transactions (much like a library) until customer-friendly, non-invasive sign-on and low cost follow-up approaches are developed to identify job-seekers in the electronic labor exchange.

      In developing these performance measures, the needs and interests of legislators, policy makers, program managers, budget planners, analysts, employers and other investors were considered. These output measures may be used for strategic planning, program management, continuous improvement and research. This information combined with customer satisfaction surveys and feedback could assist in the design of system improvements to meet customer needs.

      It is DOL's expectation that national data reporting will be produced by the AJB/ATB system, and will be aggregated and reported electronically. It is also DOL's expectation that these performance measures will not require additional state data collection.

      1. Holdings: This measure is a count of the number of employer job orders and the number of job seeker resumes in the AJB/ATB system. Continued growth in the number of job orders and resumes in the AJB/ATB system provides an indication of customer satisfaction and perceived value by the customers.

      2. Usage: This measure is a count of user sessions on the AJB/ATB system. A user is an individual who accesses the AJB/ATB system for any purpose. Some users of the system supply personal identification numbers and passwords; others are anonymous. A user session represents each single continuous access. This is different than a ``hit'', which measures the number of Internet server accesses (ie. computer to computer communications). Under this measure, we will count registered employers, registered job seekers and anonymous sessions. Enhancements to

         [[Page 32571]]

         the AJB/ATB system are underway to provide better information on anonymous sessions. This output measure will provide a gauge of how many customers are using these electronic labor exchange services over some defined period of time and whether the usage is growing, remaining static or declining.

      3. Transactions: This is an output measure of a higher level of interaction with the AJB/ATB system beyond a review of the information resources. For example, among the variety of transactions which could be collected electronically, two stand out. They are Job Seeker Referral Requests and Employer Resume Downloads.

         ``Job Seeker Referral Requests'' measures the number of job order referral instruction screens that are viewed by job seekers on AJB. This output measure is a count of the number of times job order referral instruction screens are viewed by job seekers on AJB after viewing a job order description. This measure identifies the number of users who have requested specific job referral information and are likely to respond to the employer's job opportunity. There are two ways in which this occurs: unsuppressed job orders and suppressed job orders. In unsuppressed job orders the employer contact information is broadcast and not hidden from the job seeker. In this case, the job seeker self refers. In suppressed job orders the employer contact information is not viewable or accessible through computers and the job seeker must be referred to the employer by the local labor exchange office holding the job order.

         ``Resumes Downloaded'' is a count of job seeker resumes that employers have selected from a list of job seekers, who met an employer defined search criteria, to obtain job seeker contact information. Questions

         The Department specifically invites interested persons to provide comments, data, information and views concerning the following:

         Q1. Do these proposed measures adequately represent an employer's or job seeker's satisfaction with the Electronic Labor Exchange?

         Q2. How would you rate the importance of the three types of self- service outputs (holdings, usage, and transactions)? Which would be most or least important for strategic planning, program management, etc.?

         Q3. What other ways would you suggest for measurement of self-help services?

         Q4. How do you currently or how would you suggest the national system collect customer satisfaction data and information?

         Q5. Will the proposed Self-Service Electronic Labor Exchange measures enhance State quality improvement initiatives?

         Q6. What specifically defined period of time would be most appropriate for self-service performance measurement: monthly, quarterly, or annually?

   2. For Facilitated Self-Help Strategies

      Two Facilitated Self-help measures are being proposed: the number of users and customer satisfaction. These measures are intended to determine the extent to which customers value facilitated self-help services. The measures address this by answering two questions: do customers use the service and are customers satisfied with the services? 1. Number of Users of Self-Help Service

      The Number of Users of Self-Help Service is an output measure of the number of customers who access the self-help resources which includes the assistance of a knowledgeable staff member assigned to facilitate customer access to and use of the self-help tools. It is the intent that facilitated self-help provides service to a large number of job seekers with a minimum investment of staff time.

      The accumulation of a count of users can be as rudimentary as an office stroke tally or a sign-in log. It would not be necessary to track which self-help tools are used, although valuable information needed for the continuous improvement of facilitated self-help programs might be gained from a periodic assessment of which tools are most popular or useful to job seekers. 2. Customer Satisfaction

      Customer Satisfaction can be measured by conducting periodic formal telephone or mail surveys or by collecting customer in-person feedback using a structured approach at specified intervals. The measure could capture the frequency of use of self-help resources. Customers normally are not required to register to use the self-help tools in a labor exchange office or one-stop career center. These measures recognize the customer's use of and satisfaction with self-help resources.

      Facilitated Self-Help is a widely used and valid form of service. Self-Help service is consistent with the reality of shrinking resources, and relates directly to customers' demand. This measure can contribute to continuous improvement with input from the customer.

   3. For Staff-Assisted Service Strategies

      Measures of Job Seeker Customer Services

      Staff-Assisted Labor Exchange Services are the core activities of the modern Labor Exchange Office. The purpose of these performance measures is to determine the outcomes, effectiveness, and system impact of Staff-Assisted job seeker service. Performance will be evaluated through a combination of interrelated performance measures. These measures will be useful to legislators, employers, policy makers, agency administrators and program managers since they provide a tool for managing programs, baseline data for continuous quality improvement, and feedback that will allow States to respond to customer needs. These quantitative measures should be supplemented by qualitative measures of customer satisfaction and are not intended to be the sole measures of satisfaction.

      1. Entered Employment Rate (EER): The EER is the percentage of job seekers securing employment after receiving staff-assisted labor exchange services divided by the total number of job seekers who received staff-assisted labor exchange services. This measure uses as its denominator only those customers who receive staff-assisted service(s) and not the total number of applicants registered in the labor exchange office database. This performance measure looks at the effectiveness of the Job Search Assistance services that have employment as the expected outcome. This measure encourages an increased level of job order and job seeker follow-up, tracking and employer feedback. Service and entered employment data would be collected and reported by staff to the labor exchange reporting system. Unemployment Insurance wage records can be an additional data source used to conduct job seeker follow-up on entry to employment.

         [Optional Measure] Job Development Entered Employment Rate: Is an additional and optional measure of Job Search Assistance service. Job Development (JD) has been an effective tool to help individuals who have barriers to employment or difficulty in finding employment to find jobs. With the expected expansion in the role of the Labor Exchange in Welfare-to-Work programs, it has been suggested that a JD Entered Employment Rate would be valuable for program management and continuous improvement.

         Job Development Entered Employment Rate is the percentage of job seekers who entered employment after receiving Job Development referral

         [[Page 32572]]

         compared to the total number of job seekers who received JD referral.

         An outcome measurement of the pro-active solicitation of a job opportunity for an individual or group, this measure assesses the effectiveness of job development efforts in contributing to job seeker entry to employment. Job development is defined as development of a job opening for a job seeker or group of job seekers through direct contact with potential employers when no suitable job openings are currently listed. Job development contacts are currently reported within the category of ``Received Reportable Service'' in the ETA 9002 report.

         This performance measure is intended to create an incentive for staff to use their considerable knowledge of the labor market and employers' needs to increase the number and effectiveness of job development contacts and referrals on behalf of job seekers. Measures of Employer Customer Services

         The variety of staff-assisted services that are made available to employers can be grouped into three categories: Job Listing Services, Job Matching, and Business Assistance service.

         The purpose of the proposed performance measures is to determine the extent to which employers' needs are being fulfilled in terms of the ``Timeliness,'' ``Quality,'' and ``Impact'' of our work. Staff- assisted employer service performance should be evaluated by this combination of interrelated performance measures. The individual measures should not be viewed in isolation and these measures should be supplemented by qualitative measures of employer satisfaction.

         The proposed measures provide an indirect indication of employer customer satisfaction. They are not intended to be the sole measures of satisfaction. For example, the Return (Repeat) Business measure may be a good measure of employer satisfaction at the macro (state or national) level. However, federal contractors and others who are required to list their jobs with the Agency may or may not be satisfied employer customers.

         It is recognized that services provided to employers are not currently reported to the USDOL and may not now be collected by the labor exchange agency. These employer service measures may require States to develop new administrative reporting or program record keeping systems.

      2. Job Listing Return Business Rate: The number of employers who list more than one job order divided by the total number of employers who use the Staff-Assisted Job Listing Services during a reporting year. This outcome measure of repeat business serves as a means to determine employer satisfaction with the job listing service.

         The intent of this measure is to provide an incentive for service delivery staff to improve customer service and relationships including quick response time, quality referrals, and employer follow-up contacts. Most of the data needed for the Return Business measure is readily available with the possible exception that some States do not capture employer identification numbers on job orders. This measure would require that employer identification information be collected and reported for all job orders in the agency reporting and Job Bank system.

         Some States have expressed a preference for measuring job openings rather than job orders. The focus of this measure is employer satisfaction, not the volume of job openings or the number of job orders the labor exchange receives or an employer market penetration rate. Repeat business is a better indicator of employer satisfaction, and this measure is used in the private sector as an important indicator of success in service sector business enterprises.

         Some States and labor exchange staff were also concerned with the effect of employer direct job order entry into AJB and employer access to resumes on ATB and its impact on the number of job orders they would receive. In the past the number of job orders secured by a local office and listed in the State job bank has been a key local office performance measure in some States. Again, this measure represents the percent of Return Business which results from the employer's satisfaction with the Staff-Assisted service. Job orders entered directly by employers are captured in the electronic labor exchange measures of AJB and ATB holdings and are not lost to the state. An unintended consequence of this measure may be a counter-productive competition between a State's staff-assisted service component and the State's electronic labor exchange component. From a local office viewpoint, this measure may motivate staff members to discourage employers use of the job order self entry service options in AJB to assure that there is a high rate of repeat job listing business credited to the staff assisted service component. Enhancements to the AJB/ATB systems and report generators are currently underway as mentioned above. This labor exchange system performance issue will be addressed in future AJB/ATB versions. The Department would appreciate comments and suggestions for dealing with this potential consequence of measuring Repeat Employer Business in the staff-assisted service component of the labor exchange.

         Other States commented on the impact of having a large number of small employers or adverse economic conditions which would reduce the opportunity for return business during a reporting period as short as a year. DOL acknowledges the concern and would encourage comments which are based on a review of State Employment Service, Unemployment Insurance or Tax administrative data which demonstrate this phenomenon. DOL would also be interested to learn if this is a result of an agency policy to target small employers who have limited workforce replacement or expansion needs.

         Finally, even if the suggested defect in this performance measure is accurate, the Return Business measure would be useful for strategic planning, performance management and continuous improvement efforts. This measure is intended to be used by States for national performance reporting and state program management as opposed to being used for State-to-State comparisons.

      3. Business Assistance Service Return Business: In addition to Job Listing and Job Matching services, there are a number of other types of Staff-Assisted labor market information and services which employers require. These were defined in section IV. G. above. The provision of Business Assistance Services (BAS) is intended as a means of stimulating increased employer job listings and employer support for the Labor Exchange and Workforce Development System. The BAS Return Business Rate is the number of employers who utilize more than one BAS service divided by the total number of employers who use at least one Job Listing, Job Matching or Business Assistance staff-assisted service. The intent is to measure the level of employer customer satisfaction with BAS provided and to be an incentive to increase service to an important customer.

      4. Referral Response Time: Is an outcome measure of Job Matching Service (JMS) effectiveness, which combines the interests of job- seekers and employers. The Job Order Response Time measure is the percentage of job orders for which referral-related action took place within three business days compared to the total number of job orders listed. This performance measure provides an incentive for quick response

         [[Page 32573]]

         to job orders by job-seeker referral or by direct referral-related contact with employers. Three business days have been identified as the generally accepted standard for referral and/or follow-up contact with employers who use the Job Listing Service (JLS).

         Some States indicated concerns regarding their ability to track employer follow-up contacts. It is recognized that current systems may have to be adjusted to capture this information. Although it is understood that not all job orders can have a quality referral made within three days, this measure encourages rapid response to the employer's need by the provision of other employer services such as, contact with the employer to review potential referrals, to develop a candidate recruitment strategy and/or to adjust features of the job order which prevent identification of candidates for referral, i.e., salary, experience requirements, etc.

      5. Average Time Lapse to Successful Referral: This outcome measure will provide a picture of how quickly labor exchange agencies respond to job orders with referral of a qualified candidate for employment. The performance measure is the average time lapse (in days) from the date an employer's job order is listed to the date of referral of the first agency-referred individual hired by that employer. The underlying presumption is that when the employer decides to hire one of the labor exchange referrals, the agency has met the employer's candidate qualification need.

         Not included in this calculation are job orders where the employer hires from another source. This removes from the equation situations where, although qualified candidates are referred, the employer decides to hire an equally qualified candidate from another source.

         This measure focuses on the time lapse to the date of job-seeker referral. An employer's hiring practices which may delay the offer of employment for many days or weeks does not affect the outcome of this measure. This measure provides a snapshot of the agency's referral practices in terms of responsiveness with a quality feature. It is intended to balance the rapid referral measure and mitigates the possibility that fast, but poor quality, job-seeker referrals will be the reaction to a rapid response performance measure.

         Job Order Fill Rate: Based on State comment, an additional measure of the Employer Satisfaction is proposed. The Job Order Fill Rate is the number of job orders for which a placement is made divided by the total number of suppressed job orders received within the reporting year. A job order for which the employer name, address, and other identification or contact information is hidden from the job seekers' view is considered to be a suppressed job order.

         Employers have the option of listing a job order in a suppressed or unsuppressed mode depending on the level of initial screening, and/or referral control service the employer wish the labor exchange to exert. Frequently employers are guided in the decision to suppress contact information based on the local availablity of candidates for employment who meet the skill, background and experience required to be successful candidates for employment by labor exchange staff.

         This outcome measure will provide a picture of how effectively agencies or States respond to these job listings with qualified candidates for employment. In this measure, the fact that the employer hires one of the job seekers referred by the agency is verification that the candidate was qualified for the job and that the employer was satisfied with the service provided.

         The intent of this measure is to provide an incentive to meet and/ or exceed employer customer expectations by referral of qualified candidates for employment. Not including in the denominator of this performance measure job orders which are listed in an unsuppressed or broadcast fashion avoids the possible unintended consequence that the labor exchange fill rate performance is measured against job orders where staff-assisted service is neither needed nor wanted by the businesses that want to broadcast available employment opportunities. The required data are currently being collected in many States and are available.

         In some States, a Job Opening Fill Rate performance measure has been used for many years as the approach to gauge the efficiency of local Labor Exchange offices and operations. In other States, the Job Opening Fill Rate has been discontinued because it has proven to be a disincentive to securing employer job orders in occupational areas where local management and staff would find it difficult to identify local and immediately available candidates. By including in the performance measure denominator only those job orders and openings which have the employer contact information suppressed, there will be less of an incentive to withhold job orders from the system or to artificially reduce the number of job openings. Based on labor market information, and in consultation with the employer, job orders that are not likely to be filled by a local candidate can be listed in an unsuppressed fashion and would therefore not count in the Fill Rate measure denominator. Questions

         The Department specifically invites interested persons to provide comments, data, information and views concerning the following:

         Q7. Are there other services which your State or agency provides which have entry to employment as a goal which should be included in a list of Job Search Assistance services?

         Q8. Veteran service and other target group service programs continue to provide a higher level of one-on-one job-seeker service and job development assistance. Is the level of the job development assistance activity in your State or agency significant enough to require measurement of the Job Development Entered Employment Rate separately and apart from the Staff-Assisted Entered Employment Rate (EER)?

         Q9. What would be the impact of a requirement to collect or have the facilities necessary to associate an employer ID number with a job order?

         Q10. Is the proposed three business day standard for measuring referral-related follow-up contact with employers consistent with your agency or State approach?

         Q11. The Job Order Fill Rate measure is intended to provide an indication of employer satisfaction with staff-assisted labor exchange services. Are the unintended consequences of a fill rate performance measure serious enough to eliminate it from consideration as a program performance measure?

         Q12. Are there other services and/or service delivery approaches which should be included in the mix of services suggested under the Business Assistance Service definition?

         Q13. Will these proposed Staff-Assisted Labor Exchange service measures enhance State continuous quality improvement initiatives?

   4. System Measures

      1. Cost Per Entered Employment (CPEE)

      This is an outcome measure of the efficiency of the labor exchange function. The CPEE is calculated by dividing the total number of entered employment counts by the total funds allocated to the labor exchange function. Funding sources would include Wagner-Peyser Act, Veteran Employment and Training Service (VETS) allocations to States, and supplemental State funding provided for the labor exchange function. This measure will provide the context for the public labor exchange systems to

      [[Page 32574]]

      compare its key outcome--entered employment--against other workforce development components. For example, a 40% EER that costs only $300 per entered employment may seem more favorable when compared to an 80% EER when each EE costs $5000.

      Another intent of the measure is to promote efficiency by providing management with cost-of-operation information. This information could be used for strategic planning, allocation and distribution of scarce resources, as well as continuous quality improvement efforts.

      Administrative data sources are available to produce this system measure, and includes SESA administrative records and labor exchange job-seeker service records. In many States, Unemployment Insurance wage record data can be used to improve entered employment information.

      Funding data can be captured from the annual grant allocation documents for Wagner-Peyser and Veteran's Services. The information on State funding for the labor exchange function would have to be collected by canvassing the States to determine the level of additional State support. The cost should be calculated based on the Wagner-Peyser Program Year basis, which is consistent with other employment and training program funding cycles. The Veterans' program grants would have to be prorated since they are allocated based on a Federal Fiscal Year. A similar proration would be required for State funding which is not allocated on a program year basis.

      The CPEE measure is easy to understand and is generally accepted. These data are also useful for program management and the resource allocation process. 2. Duration of Benefits Compensated

      Employers are the primary customers of Workforce Development programs nationwide. To meet employers' workforce needs, public labor exchange agencies focus on job-seekers who possess a labor market attachment and marketable skills. In addition, Wagner-Peyser Act funding is provided to the public labor exchange, in part, to administer the work test for the State Unemployment Insurance compensation system and to provide job search and placement services for claimants. The effect and/or impact of services directed toward UI benefit customers has many dimensions which can be significantly affected by movement in the economy and local business conditions. However, there is a general belief that labor exchange service can have a direct and measurable impact on the duration of benefits.

      Two options are proposed to measure the impact of staff-assisted service on unemployment insurance benefit customers:

      The first option is a measure of duration of benefits compensated for claimants who receive services compared to prior year(s)'' duration. This measure can be adjusted based on economic conditions. This measure seeks to determine the impact of staff-assisted labor exchange services on the duration of benefits for claimants who received services. Data can be collected from Unemployment Insurance average duration data records (UI claims first payments/weeks compensated), labor exchange agency records, and local office data reporting. USDOL Unemployment Actuarial Unit data on business cycles and economic conditions could also be useful in explaining the impact of job-seeker services on claimants. Benchmark data would be available from previous State reporting.

      An easy-to-understand measure of the success of labor exchange services, this measure can be adjusted to consider State and local economic conditions. Most of the required data is easily accessible. This duration-of-benefits model could also be adapted to other groups such as Welfare-to-Work customers and Veterans, etc., and could be expressed as a reduction in the number of income transfer payments, as a trust fund savings, or as a part of a return on investment statement.

      The second option is a measure of the impact of Labor Exchange Services on the Duration of Benefits Compensated for UI Claimants Required to Search for Work (Work-Test Claimants) who receive Staff- Assisted Labor Exchange services and enter employment compared to the average duration-of-benefits compensated to all claimants who are required to search for work.

      This outcome measure also seeks to determine the impact of staff- assisted labor exchange service on the duration of benefits claimants. In this option, the group to be studied will consist of UI claimants who: (1) were not exempt from an active work search; (2) received staff-assisted labor exchange job-seeker services; and (3) entered employment.

      Staff-Assisted Labor Exchange services for claimant job-seekers includes job search assistance (JSA) and a variety of facilitated self- help services enumerated in the definitions above in section IV. A, B, and C. Since the benefit customer job-seeker's identity is known, they are usually registered in the labor exchange system.

      A claimant who is required to search for work is a job-seeker who, as a condition of receiving Unemployment Insurance benefits, is required to perform an active work search. This criterion excludes UI benefit customers who are on a temporary layoff, required to secure employment through a union agent, partially unemployed and receive benefits under an approved partial benefit program, or exempt from work search requirements due to enrollment in an approved training program.

      This performance measure provides an incentive to deliver high quality JSA and other service to UI job seekers, to begin the provision of service early in the claim, to aggressively follow-up on job referrals, to track UI job-seekers' follow-through on work search plans, and to determine when the benefit claimant became re-employed.

      Administrative data are available for development of this measure of impact from Unemployment Insurance Wage Records (claims filedand average duration data), ES records (referral and service data elements in the current ETA 9002 compliant reporting systems and the Employment Security Systems Institute systems), and local office data reporting. In addition, federal and State benchmark databases which can be used to measure the State against its previous accomplishments are available.

      The measure can be used to show the effectiveness of service when a reduction in duration can be shown, and would be useful to legislators, employers, and policy makers. The impact or reduction in the duration of benefits performance measure could be:

      (a) translated into a reduction in the number of weeks compensated and a calculation of Unemployment Insurance Trust Fund savings using the State's average benefit rate, and

      (b) adapted for other target groups including Welfare-to-Work customers, Veterans, etc. In this case the result could be expressed as a reduction in the number of monthly income transfer payments, as a dollar savings in benefits using average benefit rate data, or as part of a return on investment statement. Questions

      The Department specifically invites interested persons to provide comments, data, information and views concerning the following:

      Q14. Can a measure of UI Benefit duration provide a meaningful measure of the impact of labor exchange services on those claimants who receive staff-assisted services?

      Q15. Would the measurement, and comparison of the rate of UI claim

      [[Page 32575]]

      exhaustion be a better system measure of the labor exchange staff- assisted service to UI benefit customers?

      Q16. Considering the measures proposed in this FRN, is there an optimal mix of performance measures which your State or agency would suggest?

      Finally, it should be noted that the Workforce Development legislation currently pending before the Congress (H.R. 1385 and S. 1186) provides core performance measures that include, in addition to entered employment, retention in employment six months after entry and increases in earnings. Under the proposed legislation, these measures would apply to participants in the Workforce Development activities provided through the One-stop System, including labor exchange activities.

      We would appreciate receiving views on the appropriateness of these additional measures for measuring public labor exchange programs.

      Paperwork Reduction Act

      The notice issued here is not subject to the requirements of the Paperwork Reduction Act of 1995 (44 U.S.C. 3501 et seq.) Because it contains no ``collection of information'' as defined in 44 U.S.C. 3502(3).

      Signed at Washington, D.C., this 8th day of June 1998. Raymond J. Uhalde, Acting Assistant Secretary for Employment and Training.

      BILLING CODE 4510-30-P

      [[Page 32576]]

      [GRAPHIC] [TIFF OMITTED] TN12JN98.016

      [[Page 32577]]

      [GRAPHIC] [TIFF OMITTED] TN12JN98.017

      [[Page 32578]]

      [GRAPHIC] [TIFF OMITTED] TN12JN98.018

      [FR Doc. 98-15754Filed6-11-98; 8:45 am]

      BILLING CODE 4510-30-C