77 FR 115 pgs. 35665-35666 - Notice of Proposed Information Collection Requests; Office of Planning, Evaluation and Policy Development; Study of Strategies for Improving the Quality of Local Grantee Program Evaluation
Type: NOTICEVolume: 77Number: 115Pages: 35665 - 35666
FR document: [FR Doc. 2012-14593 Filed 6-13-12; 8:45 am]
Agency: Education Department
Official PDF Version: PDF Version
DEPARTMENT OF EDUCATION
Notice of Proposed Information Collection Requests; Office of Planning, Evaluation and Policy Development; Study of Strategies for Improving the Quality of Local Grantee Program Evaluation
SUMMARY:
This study is intended to inform the Department's decisions about how to structure future grant competitions; how to support evaluation and performance reporting activities among funded grantees, including technical assistance to improve the quality of evaluations and performance reporting; and how to make the best possible use of grantee evaluation findings.
DATES:
Interested persons are invited to submit comments on or before August 13, 2012.
ADDRESSES:
Written comments regarding burden and/or the collection activity requirements should be electronically mailed to ICDocketMgr@ed.gov or mailed to U.S. Department of Education, 400 Maryland Avenue SW., LBJ, Washington, DC 20202-4537. Copies of the proposed information collection request may be accessed from http://edicsweb.ed.gov , by selecting the "Browse Pending Collections" link and by clicking on link number 04869. When you access the information collection, click on "Download Attachments" to view. Written requests for information should be addressed to U.S. Department of Education, 400 Maryland Avenue SW., LBJ, Washington, DC 20202-4537. Requests may also be electronically mailed to ICDocketMgr@ed.gov or faxed to 202-401-0920. Please specify the complete title of the information collection and OMB Control Number when making your request.
Individuals who use a telecommunications device for the deaf (TDD) may call the Federal Information Relay Service (FIRS) at 1-800-877-8339.
SUPPLEMENTARY INFORMATION:
Section 3506 of the Paperwork Reduction Act of 1995 (44 U.S.C. Chapter 35) requires that Federal agencies provide interested parties an early opportunity to comment on information collection requests. The Director, Information Collection Clearance Division, Privacy, Information and Records Management Services, Office of Management, publishes this notice containing proposed information collection requests at the beginning of the Departmental review of the information collection. The Department of Education is especially interested in public comment addressing the following issues: (1) Is this collection necessary to the proper functions of the Department; (2) will this information be processed and used in a timely manner; (3) is the estimate of burden accurate; (4) how might the Department enhance the quality, utility, and clarity of the information to be collected; and (5) how might the Department minimize the burden of this collection on the respondents, including through the use of information technology. Please note that written comments received in response to this notice will be considered public records.
Title of Collection: Study of Strategies for Improving the Quality of Local Grantee Program Evaluation.
OMB Control Number: 1875-NEW.
Type of Review: New.
Total Estimated Number of Annual Responses: 20.
Total Estimated Number of Annual Burden Hours: 20.
Abstract: The U.S. Department of Education provides support to states, districts and schools through a number of competitive and formula grant programs. Through these programs, the federal government funds a wide array of activities, from professional development for teachers to turnaround efforts for failing schools. High-quality evaluation plays an essential role informing policy makers about program performance, outcomes and impact. Performance reporting with high-quality data can indicate whether a funded project is meeting its goals and taking place as planned. Project evaluations can explore how best to implement a particular educational practice, whether positive student outcomes were attained, or whether a particular educational intervention actually caused the outcomes observed.
To date, the Department lacks comprehensive information about the quality or rigor of the performance reporting and evaluation activities its grantees are undertaking and whether the technical assistance provided has been useful in improving the quality of the performance reporting or evaluations. Accordingly, the focus of this study is to examine the influence of Department-funded technical assistance practices on the quality and rigor of grantee evaluations and performance reporting in two Department programs (described below). It will describe the technical assistance provided by the Department to support grantee performance reporting and evaluation; explore how grantees perceive the technical assistance has influenced their activities; assess the quality of performance reporting and evaluations undertaken; and determine how the findings from performance reporting and evaluations were used both by grantees and by the Department.
This study will be based upon a systematic review of existing documentation as well as interviews with selected grantees and with federal staff and federal contractors involved in grant monitoring and in the provision of technical assistance to grantees. The interviews with selected grantees are the subject of this OMB clearance request.
This study will focus on two grant programs within the Department's Office of Innovation and Improvement: the Charter Schools Program: State Educational Agencies (CSP SEA) program and the Voluntary Public School Choice (VPSC) program. A brief description of each program is provided below.
1. Voluntary Public School Choice (authorized under the Elementary and Secondary Education Act of 1965, as amended, Title V, Part B, Subpart 3, (20 U.S.C. 7225-7225g)). The goal of the VPSC program is to support the creation and development of a large number of high-quality charter schools that are held accountable for enabling students to reach challenging state performance standards, and are open to all students. The program was first enacted as part of the No Child Left Behind Act of 2001 (Pub. L. No. 107-110, § 115, Stat. 1425) to support the emergence and growth of choice initiatives across the country. VPSC's goal is to assist states and local school districts in creating, expanding, and implementing public school choice programs. The program has awarded two cycles of competitive grants to states, local education agencies, and partnerships that include public, nonprofit and for-profit organizations. In 2007, the most recent award year, the program awarded a total of 14 competitive grants to two states, eight school districts, a charter school, an intermediate school district, and KIPP schools in Texas.
2. Charter Schools Program: State Educational Agencies (authorized under the Elementary and Secondary Education Act of 1965, Section 5201-5211 (20 U.S.C. 7221a)). Federal support for charter schools began in 1995 with the authorization of the CSP. The CSP SEA program awards competitive grants to state education agencies to plan, design, and implement new charter schools, as well as to disseminate information on successful charter schools. The key goals of the CSP SEA program are to increase the number of charter schools in operation across the nation and to increase the number of students who are achieving proficiency on state assessments of math and reading. The CSP statute also addresses expanding the number of high-quality charter schools and encouraging states to provide support for facilities financing equal to what states provide for traditional public schools. Grants have been awarded to 40 states, including awards to 33 states since 2005. Grants are typically awarded for three years and may be renewed.
This study will review all technical assistance provided to CSP SEA and VPSC grantees on performance reporting and evaluations and how grantees conduct these activities. All CSP SEA and VPSC grantees are required by the Department to conduct performance reporting. Although grantees are not required to conduct any particular type of evaluation, the study will review both impact evaluations and non-impact evaluations conducted by grantees. The study approach, with respect to the review of performance reporting, impact evaluations, and non-impact evaluations, is described below.
Performance Reporting
The goal of performance reporting is to measure performance and track outcomes of the project's stated goals and objectives. The collection of accurate data on program performance is necessary for the reporting required by the Government Performance and Results Act (GPRA), which was passed by Congress in 1993 and updated through the GPRA Modernization Act of 2010. The latter will require even more frequent reporting-quarterly instead of annually.
In addition to requiring grantees to collect annual data in support of GPRA reporting, the CSP SEA and VPSC programs encourage grantees to develop implementation and outcome measures in support of other program goals. Throughout this document, when we refer to grantee performance measures, we are referring to the measures grantees use not only for GPRA reporting, but also for reporting on other activities and outcomes.
The CSP and VPSC programs provide technical assistance to grantees on developing appropriate objectives and performance measures and on obtaining quality data in support of those measures. Because all grantees conduct some kind of performance reporting, this study's examination of performance reporting encompasses all grantees. It will describe the type of technical assistance provided, categorize the types of performance measures that grantees address, determine whether the measures are responsive to the GPRA indicators defined for each program, review whether the initial set of performance measures changed as a result of the technical assistance received, and examine the quality and appropriateness of data collection for those measures.
Impact Evaluation
While documenting implementation activities and outcomes can be useful to school and district administrators, it does not provide information on the effectiveness of funded interventions. The only evaluation designs that provide credible evidence about the impacts of interventions are rigorous experimental and quasi-experimental designs. Impact evaluations can provide guidance about what interventions should be considered for future funding and replication.
This study will review the quality and rigor of all impact evaluations being conducted of higher-order outcomes, particularly student achievement, using criteria that were adapted from the What Works Clearinghouse review standards for grantees as part of the Department's Data Quality Initiative. These criteria were revised by Abt Associates as part of its annual review of Mathematics and Science Partnership final-year evaluations. The criteria are listed in Appendix A. The study will also examine the completeness and clarity of evaluation reports submitted as part of an impact evaluation.
Non-Impact Evaluation
Grantees may choose to conduct non-impact evaluations to examine program outcomes and implementation processes. Non-impact evaluations may include both formative implementation and process evaluations that evaluate a program as it is unfolding, and summative descriptive evaluations that examine changes in final outcomes in a non-causal manner. A full framework of formative and summative evaluations is included in Appendix B.
The main focus of the review of non-impact evaluations will be on those that focus on a change in higher-order outcomes using a one-group pre-post design. For these evaluations, the study will examine the appropriateness of data collection strategies for the design chosen and whether the findings of the study are described appropriately based on the design. The study will also describe other non-impact evaluations that grantees have undertaken, without commenting on their quality.
Data collection, including conducting interviews and reviewing extant documents, is required to complete this study. Part A of this request discusses the justification for these data collection activities, while Part B describes the data collection and analysis procedures.
Darrin A. King,
Director, Information Collection Clearance Division, Privacy, Information and Records Management Services, Office of Management.
[FR Doc. 2012-14593 Filed 6-13-12; 8:45 am]
BILLING CODE 4000-01-P