Request for Information To Gather Technical Expertise Pertaining to Testing Integrity

Summary

---

In light of recent, high-profile reports of misconduct by school officials in the test administration process, the U.S. Department of Education (``the Department'' or ``we'') is seeking to collect and share information about best practices that have been used to prevent, detect, and respond to irregularities in academic testing. To that end, the Department is taking several steps, described below, to collect information and gather suggestions to assist State educational agencies (SEAs), local educational agencies (LEAs), and the testing-integrity-focused organizations that service them. The Department anticipates making use of this information to facilitate further dialogue and to help SEAs and LEAs identify, share, and implement best practices for preventing, detecting, and investigating irregularities in academic testing. First, the Department is issuing this request for information (RFI) to collect information about the integrity of academic testing. We pose a series of questions to which we invite interested members of the public to respond. Second, the Department will host a symposium where external experts can engage in further discussion and probe these issues in greater depth. Third, the Department will publish a document that contains a summary of the recommendations that were developed as a result of the RFI and the symposium, as well as other resources identified by external experts participating in the symposium.

Text

---

 

Federal Register, Volume 77 Issue 10 (Tuesday, January 17, 2012)

Federal Register Volume 77, Number 10 (Tuesday, January 17, 2012)

Notices

Pages 2280-2282

From the Federal Register Online via the Government Printing Office www.gpo.gov

FR Doc No: 2012-753

-----------------------------------------------------------------------

DEPARTMENT OF EDUCATION

Docket ID: ED-2012-OESE-0003

Request for Information To Gather Technical Expertise Pertaining to Testing Integrity

AGENCY: Office of Elementary and Secondary Education, Department of Education.

ACTION: Request for information.

-----------------------------------------------------------------------

SUMMARY: In light of recent, high-profile reports of misconduct by school officials in the test administration process, the U.S. Department of Education (``the Department'' or ``we'') is seeking to collect and share information about best practices that have been used to prevent, detect, and respond to irregularities in academic testing. To that end, the Department is taking several steps, described below, to collect information and gather suggestions to assist State educational agencies (SEAs), local educational agencies (LEAs), and the testing-integrity-focused organizations that service them. The Department anticipates making use of this information to facilitate further dialogue and to help SEAs and LEAs identify, share, and implement best practices for preventing, detecting, and investigating irregularities in academic testing.

First, the Department is issuing this request for information (RFI) to collect information about the integrity of academic testing. We pose a series of questions to which we invite interested members of the public to respond. Second, the Department will host a symposium where external experts can engage in further discussion and probe these issues in greater depth.

Third, the Department will publish a document that contains a summary of the recommendations that were developed as a result of the RFI and the symposium, as well as other resources identified by external experts participating in the symposium.

DATES: Written submissions must be received by the Department on or before 5 p.m., Washington, DC time, on February 16, 2012.

ADDRESSES: Submit your comments through the Federal eRulemaking Portal or via postal mail, commercial delivery, or hand delivery. We will not accept comments by fax or by email. To ensure that we do not receive duplicate copies, please submit your comments only one time. In addition, please include the Docket ID and the term ``Testing Integrity response'' at the top of your comments.

Federal eRulemaking Portal: Go to http://www.regulations.gov to submit your comments electronically. Information on using Regulations.gov, including instructions for accessing agency documents, submitting comments, and viewing the docket, is available on the site under ``How To Use This Site.''

Postal Mail, Commercial Delivery, or Hand Delivery: If you mail or deliver your comments, address them to Carlos Martinez, Office of Elementary and Secondary Education, Attention: Testing Integrity RFI, U.S. Department of Education, 400 Maryland Avenue SW., room 3W104, Washington, DC 20202-6132.

Privacy Note: The Department's policy for comments received from members of the public (including comments submitted by mail, commercial delivery, or hand delivery)

Page 2281

is to make these submissions available for public viewing in their entirety on the Federal eRulemaking Portal at http://www.regulations.gov. Therefore, commenters should be careful to include in their comments only information that they wish to make publicly available on the Internet.

Given the subject matter, some comments may include proprietary information as it relates to confidential commercial information. The Freedom of Information Act defines ``confidential commercial information'' as information the disclosure of which could reasonably be expected to cause substantial competitive harm. You may wish to request that we not disclose what you regard as confidential commercial information.

To assist us in making a determination on your request, we encourage you to identify any specific information in your comments that you consider confidential commercial information. Please list the information by page and paragraph numbers.

FOR FURTHER INFORMATION CONTACT: Carlos Martinez, U.S. Department of Education, 400 Maryland Avenue SW., Room 3W104, Washington, DC 20202-

6132 by phone at (202) 260-1440.

If you use a telecommunications device for the deaf (TDD), call the Federal Relay Service (FRS), toll free, at 1-(800) 877-8339.

SUPPLEMENTARY INFORMATION:

1. Introduction

The Department is seeking information about testing integrity that may help SEAs and LEAs ensure the integrity of the data used to measure student achievement and to ensure meaningful educational accountability in SEAs and LEAs. This is a request for information only. This RFI is specifically inquiring into best practices regarding: (1) Preventing and reducing testing irregularities in State academic assessments; (2) detecting and analyzing testing irregularities; (3) reviewing and investigating alleged testing irregularities; and (4) for assessments that are increasingly delivered online and by computer, how responses to the first three issues described above might be different from those that apply to assessments administered through more traditional means.

For the purposes of this RFI, a testing irregularity includes any occurrence that may inappropriately influence a student's performance on a State academic assessment, provide the appearance of impropriety, or otherwise constitute a breach in test security or improper administration of State academic testing.

This RFI is issued solely for information and planning purposes and is not a request for proposals (RFP) or a promise to issue an RFP or a notice inviting applications (NIA). This RFI does not commit the Department to contract for any supply or service whatsoever. Further, the Department is not now seeking proposals and will not accept unsolicited proposals. The Department will not pay for any information or administrative costs that you may incur in responding to this RFI.

If you do not respond to this RFI, you may still apply for future contracts and grants. The Department posts RFPs on the Federal Business Opportunities Web site (http://www.fbo.gov). The Department announces grant competitions in the Federal Register (http://www.gpo.gov/fdsys). It is your responsibility to monitor these sites to determine whether the Department issues an RFP or NIA after considering the information received in response to this RFI.

The documents and information submitted in response to this RFI become the property of the U.S. Government and will not be returned.

2. Background

Educators, parents, and the public in general rely on accurate, reliable, and timely information on student academic performance to improve instruction and help all students reach and maintain high levels of achievement. Indeed, the availability of valid, reliable, and timely data on student performance is essential in informing instruction, identifying professional development needs, helping ensure meaningful accountability, and implementing effective education reforms.

Accordingly, SEAs and LEAs must ensure the integrity of the data they use to measure student achievement and ensure meaningful educational accountability. Under section 1111(b)(3)(C)(iii) of the Elementary and Secondary Education Act of 1965, as amended, (ESEA) and 34 CFR 200.1-200.24, States must establish and maintain assessment systems that are valid, reliable, and consistent with nationally recognized professional and technical standards. Even the slightest appearance of impropriety in the test administration process can undermine State accountability systems--painstakingly built over the past decade--and damage the credibility of reform efforts underway across the country.

Accordingly, State and local officials have an interest in, and share responsibility for defending against, security breaches and threats to educational data integrity. States have a long history of stewardship of academic assessments, and many States have made great efforts to ensure that their assessments and other data collection instruments are properly administered and that data security requirements are clearly specified and followed.

The Department also has a role in this area. Under Title I of the ESEA, the Department is required to review and approve each State's assessment system. Accordingly, the Department examines evidence compiled and submitted by each State about its process for monitoring and improving the technical quality of its system. During the review of State assessment systems, the Department specifically examines procedures and policies for test security and data quality, including the training and monitoring of staff.

For these reasons, this RFI seeks solutions; advice; technical information; legal, regulatory, and policy approaches; and other input from the public regarding best practices for the prevention, detection, and investigation of alleged or actual testing irregularities. Through this RFI, the Department also seeks to gather information and suggestions for SEAs and LEAs on how they can address these issues.

In addition, as noted earlier, the Department will host a symposium where external experts can engage in further discussion and probe these issues in greater depth. Responses to the RFI will be shared with the external experts, to inform their planning for the symposium. A summary of the recommendations that are developed as a result of the RFI and the symposium, as well as other resources identified by external experts participating in the symposium, will be published shortly thereafter to help inform the field.

3. Context for Responses

3.1 The primary goal of this RFI is to gather information that will help SEAs and LEAs better understand existing best practices for preventing, detecting, and investigating testing irregularities. To that end, the Department welcomes responses that address SEA and LEA policies and practices related to these areas and to State laws and regulations. To help focus our consideration of the responses provided, we have developed several questions. Because the questions are only guides to helping us better understand the issues surrounding testing integrity, respondents do not have to respond to any specific question, and may provide comments in a format that is convenient to them. Commenters may also provide relevant

Page 2282

information that is not responsive to a particular question but may, nevertheless, be helpful.

3.2 Questions Regarding the Prevention of Testing Irregularities.

3.2.1 Best Practices and Policies. Describe the best practices and policies that SEAs and LEAs have implemented to prevent testing irregularities. What evidence exists that these are best practices? Where have these best practices been adopted? What are the general lessons learned from those adoptions? How might such best practices be effectively used in the future? Are there barriers to the adoption of these best practices at the SEA or LEA level? What controls are most effective in preventing testing irregularities?

3.2.2 School Culture. What role does school culture play in test security? For example, how has professional development been used to train school officials to help prepare students and parents for academic testing? What are SEAs and LEAs doing to ensure that educators are prepared? Are SEAs providing sufficient information to LEAs about their expectations regarding the integrity of academic testing? Have the consequences for misconduct during the testing process been clearly communicated to school officials?

3.2.3 Contractual Provisions. For those States that have assessment contracts, what provisions are included in these contracts to help prevent testing irregularities? What contractual provisions have been effective in preventing testing irregularities? What evidence exists that these provisions are effective? What provisions have States included in their quality assurance contracts to help analyze risks?

3.2.4 Federal, State, and Local Roles. What are the most appropriate roles for the Department, SEAs, and LEAs in preventing testing irregularities?

Questions Regarding the Detection of Testing Irregularities

3.2.5 Detection Analyses. How are testing irregularities generally detected? What are the different types of analyses that can be used to detect testing irregularities? What are the best practices and policies that SEAs and LEAs have used to detect testing irregularities? What is each type of analysis used for? How should the results of these analyses be interpreted? Can different types of analyses be used in conjunction with one another or to complement one another? What evidence exists that these are best practices? What is the appropriate sequence of events when seeking to determine whether testing irregularities have occurred? Specifically, what steps should be taken and in what order?

Questions Regarding the Review and Investigation of Alleged Testing Irregularities

3.2.6 Contractual Provisions. What provisions have States included in their assessment contracts to help detect irregularities (e.g., provisions related to the use of high-quality control plans)? What contractual provisions have been most effective in detecting testing irregularities?

3.2.7 Federal, State, and Local Roles. What are the appropriate roles for the Department, SEAs, and LEAs in responding to allegations of testing irregularities? Who are the parties involved in an investigation at the SEA and LEA levels?

3.2.8 Responses to Alleged Testing Irregularities. If testing irregularities are detected, what are the best practices for investigating them? What forensic analyses should be used? What cooperative practices between SEAs and LEAs have yielded positive outcomes? What are barriers to investigating testing irregularities?

3.2.9 Managing Wrongdoing. If alleged testing irregularities are a result of wrongdoing, under what circumstances is it appropriate to impose strict and meaningful sanctions against wrongdoers? Are educators subject to standards of professional conduct, laws, or regulations that dictate the type of sanctions that might be imposed on an individual who violates the law or compromises professional standards? How should intent of wrongdoing be determined, and by what entity? What can be done to restore the credibility of a school system that has been tarnished by alleged or actual wrongdoing?

Questions Regarding Online and Computer-Based Assessments

3.2.10 Changes in Technology. In a world where academic assessments are increasingly delivered online and by computer, how do responses to the questions listed above change when applied to online and computer-

based assessments?

3.2.11 Computer-based Assessment Protection. What mechanisms or processes exist to ensure that the results of computer-based assessments are accurate and free from tampering? What are the best practices and policies that SEAs and LEAs have implemented in this area? What evidence exists that these are best practices? What are the potential threats to, and weak points in, computer-based assessment systems? Where are there likely opportunities for tampering and testing irregularities within the context of computer-based assessments?

Accessible Format: Individuals with disabilities can obtain this document in an accessible format (e.g., Braille, large print, audiotape, or compact disc) on request to the program contact person listed under FOR FURTHER INFORMATION CONTACT.

Electronic Access to This Document

The official version of this document is the document published in the Federal Register. Free Internet access to the official edition of the Federal Register and the Code of Federal Regulations is available via the Federal Digital System at: www.gpo.gov/fdsys. At this site you can view this document, as well as all other documents of this Department published in the Federal Register, in text or Adobe Portable Document Format (PDF). To use PDF you must have Adobe Acrobat Reader, which is available free at the site.

You may also access documents of the Department published in the Federal Register by using the article search feature at: www.federalregister.gov. Specifically, through the advanced search feature at this site, you can limit your search to documents published by the Department.

Program Authority: 20 U.S.C. 6771.

Dated: January 11, 2012.

Michael Yudin,

Acting Assistant Secretary for Elementary and Secondary Education.

FR Doc. 2012-753 Filed 1-13-12; 8:45 am

BILLING CODE 4000-01-P

---