78 FR 17 pgs. 5337-5345 - Proposed Priorities, Requirements, Definitions, and Selection Criteria—Enhanced Assessment Instruments
Type: PRORULEVolume: 78Number: 17Pages: 5337 - 5345
Docket number: [Docket ID ED-2012-OESE-0033]
FR document: [FR Doc. 2013-01567 Filed 1-24-13; 8:45 am]
Agency: Education Department
Official PDF Version: PDF Version
DEPARTMENT OF EDUCATION
34 CFR Chapter II
[Docket ID ED-2012-OESE-0033]
Proposed Priorities, Requirements, Definitions, and Selection Criteria-Enhanced Assessment Instruments
AGENCY:
Office of Elementary and Secondary Education, Department of Education.
ACTION:
Proposed priorities, requirements, definitions, and selection criteria.
Catalog of Federal Domestic Assistance (CFDA) Number: 84.368
SUMMARY:
The Assistant Secretary for Elementary and Secondary Education proposes priorities, requirements, definitions, and selection criteria under the Enhanced Assessment Instruments Grant program, also called the Enhanced Assessment Grants (EAG) program. The Assistant Secretary may use one or more of these priorities, requirements, definitions, and selection criteria for competitions using funds from fiscal year (FY) 2012 and later years. The Department takes these actions in order to establish priorities, requirements, definitions, and selection criteria that are likely to recognize high-quality proposals and to help focus Federal financial assistance on the pressing needs of, and promising developments in, developing or enhancing assessments under the Elementary and Secondary Education Act of 1965, as amended (ESEA).
DATES:
We must receive your comments on or before February 25, 2013, and we encourage you to submit comments well in advance of this date.
ADDRESSES:
Submit your comments through the Federal eRulemaking Portal or via postal mail, commercial delivery, or hand delivery. We will not accept comments by fax or by email. To ensure we do not receive duplicate comments, please submit your comments only once. In addition, please include the Docket ID and the term "Enhanced Assessment Grants-Comments" at the top of your comments.
Federal eRulemaking Portal: Go to www.regulations.gov to submit your comments electronically. Information on using Regulations.gov, including instructions for accessing agency documents, submitting comments, and viewing the docket, is available on the site under "How To Use This Site."
Postal Mail, Commercial Delivery, or Hand Delivery: If you mail or deliver your comments about these proposed priorities, requirements, definitions, and selection criteria, address them to the Office of Elementary and Secondary Education (Attention: Enhanced Assessment Grants-Comments), U.S. Department of Education, 400 Maryland Avenue SW., room 3w110, Washington, DC 20202-6132.
Privacy Note:
The Department's policy is to make all comments received from members of the public available for public viewing in their entirety on the Federal eRulemaking Portal at www.regulations.gov. Therefore, commenters should be careful to include in their comments only information that they wish to make publicly available.
FOR FURTHER INFORMATION CONTACT:
Erin Shackel. Telephone: (202) 453-6423 or by email: erin.shackel@ed.gov.
If you use a telecommunications device for the deaf (TDD), call the Federal Relay Service (FRS), toll free, at 1-800-877-8339.
SUPPLEMENTARY INFORMATION:
Invitation to Comment: We invite you to submit comments regarding this notice. To ensure that your comments have maximum effect in developing the notice of final priorities, requirements, definitions, and selection criteria, we urge you to identify clearly the specific proposed priority, requirement, definition, or selection criterion that each comment addresses.
Please note that we have included existing requirements and selection criteria in this document to provide context and to make it easier to comment on the requirements and selection criteria we are proposing. We seek comment only on the proposed priorities, requirements, definitions, and selection criteria.
We invite you to assist us in complying with the specific requirements of Executive Order 12866 and its overall requirement of reducing regulatory burden that might result from these proposed priorities, requirements, definitions, and selection criteria. Please let us know of any further ways the Department could reduce potential costs or increase potential benefits while preserving the effective and efficient administration of the program.
During and after the comment period, you may inspect all public comments about this notice in room 3W110, 400 Maryland Avenue SW., Washington, DC, between the hours of 8:30 a.m. and 4:00 p.m., Washington, DC time, Monday through Friday of each week except Federal holidays.
Assistance to Individuals with Disabilities in Reviewing the Rulemaking Record: On request the Department will provide an appropriate accommodation or auxiliary to aid an individual with a disability who needs assistance to review the comments or other documents in the public rulemaking record for this notice. If you want to schedule an appointment for this type of accommodation or auxiliary aid, please contact the person listed under FOR FURTHER INFORMATION CONTACT .
Purpose of Program: The purpose of the EAG program is to enhance the quality of assessment instruments and systems used by States for measuring the academic achievement of elementary and secondary school students.
Program Authority:
20 U.S.C. 7301a.
Proposed Priorities:
This notice contains two proposed priorities. The Department may apply one or more of these priorities in any year in which a competition for program funds is held.
Background:
Section 6112 of the ESEA authorizes the Department, through the EAG program, to make competitive grant awards to State educational agencies (SEAs) and consortia of SEAs to help them enhance the quality of their assessment instruments and assessment systems. The EAG program includes the following four statutory priorities:
(a) Collaborating with institutions of higher education, other research institutions, or other organizations to improve the quality, validity, and reliability of State academic assessments beyond the requirements for such assessments described in section 1111(b)(3) of the ESEA;
(b) Measuring student academic achievement using multiple measures of student academic achievement from multiple sources;
(c) Charting student progress over time; and
(d) Evaluating student academic achievement through the development of comprehensive academic assessment instruments, such as performance- and technology-based academic assessments.
An applicant for EAG funds must address one or more of these statutory priorities to be eligible for an award.
Through this notice, the Department proposes two additional priorities that are designed to support States' assessment work in early learning. The Department believes that a high-quality State early learning system involves several key components. These include, among other elements, early learning and development standards (as defined in this notice) that reflect the essential domains of school readiness (as defined in this notice) and a comprehensive early learning assessment system (as defined in this notice). Such an assessment system, when well-designed and properly implemented, can inform teaching and program improvement and contribute to better outcomes for children.1
Footnotes:
1 National Research Council (2008). Early Childhood Assessment: Why, What, and How. Committee on Developmental Outcomes and Assessments for Young Children, C.E. Snow and S.B. Van Hemel, Editors. Board on Children, Youth, and Families, Board on Testing and Assessment, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. Available at www.nap.edu/catalog.php?record_id=12446.
The priorities we propose in this notice focus on one piece of a comprehensive early learning assessment system-the kindergarten entry assessment (KEA). In particular, these priorities will support the development or enhancement of KEAs and promote collaboration among States in the development or enhancement of a common KEA.
A KEA is a critical piece of a comprehensive early learning assessment system because it provides a snapshot of children's learning and development at kindergarten entry. A well-designed and properly implemented KEA also can provide data to suggest areas where children may need interventions or additional supports in order to be successful in the early grades. Over time, when included as part of a comprehensive early learning assessment system, a KEA can provide data that will inform State efforts to improve child learning outcomes and help close achievement gaps.
Over the last decade, States have demonstrated an increased interest in understanding children's learning and development at kindergarten entry. Approximately half of States have instituted some form of early learning assessment.2However, these assessments vary widely in their alignment with early learning and development standards, in the depth and scope of the domains they address, and in how the data generated are used.3
Footnotes:
2 J. Stedron & A. Berger. 2010. NCSL Technical Report: State Approaches to School Readiness Assessment (updated August 2010). Denver, CO: National Conference of State Legislators. www.ncsl.org/documents/Educ/KindergartenAssessment.pdf.
3 S. Daily, M. Burkhauser, & T. Halle. 2010. "A Review of School Readiness Practices in the States: Early Learning Guidelines and Assessments." Early Childhood Highlights 1 (3). Available at www.childtrends.org/Files/Child_Trends-2010_06_18_ECH_SchoolReadiness.pdf.
The priorities proposed in this notice build on the Department's efforts to fund States collaborating to support children and youth across the cradle-through-college-to-career continuum. Grants under three Department programs, including the EAG program, currently support State-led efforts to develop common assessments among States. The Department has funded two EAG awards to support States collaborating to develop English language proficiency (ELP) assessment systems. The assessments in the systems developed under these EAG-ELP grants must be aligned with English language proficiency standards that correspond to a common set of college- and career-ready standards in English language arts and mathematics. The Department also is funding projects involving large consortia of States through the Race to the Top Assessment (RTTA) program and companion projects through the General Supervision Enhancement Grants (GSEG) program under the Individuals with Disabilities Education Act (IDEA) to develop both general and alternate assessments that are aligned with a common set of college- and career-ready standards in English language arts and mathematics.
In addition, the Department is maintaining support for the beginning of the cradle-through-college-to-career continuum through the Race to the Top-Early Learning Challenge (RTT-ELC) program. Jointly administered with the U.S. Department of Health and Human Services, RTT-ELC reflects the Departments' commitment to supporting America's youngest learners in developing the knowledge, skills, and dispositions toward learning they need to enter kindergarten ready to succeed in school and in life. To date, 14 States have been awarded RTT-ELC grants to fund education reform through developing or enhancing coordinated State systems of early learning. These RTT-ELC grants specifically support States' efforts to increase the number of children with high needs enrolled in high-quality early learning and development programs.
Recipients of RTT-ELC grants are eligible to apply for grants under the EAG program, including competitions (if any) using the KEA priority. However, the Department expects that these applicants will propose activities that are consistent with but do not duplicate activities included in their RTT-ELC applications.
Proposed Priority 1-Kindergarten Entry Assessment
Background: The Department believes that a high-quality KEA should provide critical information about children's learning and development across all the essential domains of school readiness (as defined in this notice), inform instruction at kindergarten entry and throughout the year, and support efforts to close the school-readiness gap. Families should be able to use this information to provide support for children at home. Teachers should be able to use this information to modify instruction at kindergarten entry and throughout the year, adapt curricula, and focus professional development needs. In addition, a high-quality KEA should provide information to support effective programmatic decisions and better target investments in the years before kindergarten. Proposed Priority 1 would support the development or enhancement of high-quality KEAs. These assessments would be integrated into States' student assessment systems and, if they exist, into the States' early learning assessment systems.
Under the proposed priority a KEA would be administered to children soon enough after their enrollment in kindergarten so that results could be used to inform instruction at kindergarten entry and throughout the year, adapt curricula, and focus professional development to help close any educational gaps.
The proposed priority also would require that the KEA be aligned with States' high-quality early learning and development standards (as defined in this notice), which are aligned with the States' K-3 academic content standards in, at a minimum, early literacy and mathematics. In addition, KEAs developed under the proposed priority must measure each child's development across the full range of the essential domains of school readiness (as defined in this notice).
A KEA developed or enhanced under this proposed priority must be of high technical quality and be consistent with the guidelines on early childhood assessments made by the National Research Council.4We propose to require that these KEAs be consistent with the National Research Council guidelines in light of the direction we received from Congress for the RTT-ELC program that States receiving grants under that program provide an assurance that any use of early childhood assessments conform to National Research Council reports on early childhood.5We believe that Congress would also expect that any early learning assessments developed under the EAG program would be similarly aligned with the National Research Council findings.
Footnotes:
4 See www.nap.edu/catalog.php?record_id=12446.
5 See Department of Defense and Full-Year Continuing Appropriations Act, 2011, Division B, § 1832(b), Public Law 112-10 (April 15, 2011).
Further, a KEA developed or enhanced under this proposed priority must not be used to prevent children's entry into kindergarten.
In short, the proposed priority is intended to produce KEAs that provide a snapshot of information on children's learning and development across multiple domains and can be integrated into States' student assessment systems, and if they exist, included in a States' comprehensive early learning assessment systems. The data generated from a KEA developed or enhanced through this grant would inform and support educators in providing effective learning opportunities to every child, and prevent or close achievement gaps.
Proposed Priority 1: Kindergarten Entry Assessment.
To meet this priority, an applicant must propose a project that supports the development or enhancement of a KEA that meets the following requirements:
(a) Purpose. The KEA must-
(1) Yield information that enables State and local agencies to effectively target investments for early learning and development systems serving children in the years before kindergarten;
(2) Yield information that enables programmatic decision-making at the school level, such as identifying individual children's needs and providing necessary supports to children and teachers in order to meet those needs at kindergarten entry and throughout the year;
(3) Yield information to guide individualized instruction for children enrolled in kindergarten and throughout the school year;
(4) Provide families with information about their children's learning and development based on the essential domains of school readiness (as defined in this notice); and
(5) Not be used to prevent children's entry into kindergarten.
(b) Design. The KEA must-
(1) Be a component of a State's student assessment system, including, a State's comprehensive early learning assessment system (as defined in this notice) for each State included in an application in which a comprehensive early learning assessment system exists;
(2) Be aligned with a set of early learning and development standards (as defined in this notice);
(3) Measure the full range of learning and development across the essential domains of school readiness (as defined in this notice);
(4) Measure children's learning and development against a set of levels of performance where the levels of performance encompass descriptors of what a child knows and is able to do for each level, are common statewide, and, if the applicant State applies on behalf of a consortium, are common across States in the consortium;
(5) Provide a summative assessment of each child's learning and development at kindergarten entry across the essential domains of school readiness (as defined in this notice);
(6) Be capable of assessing all children in the applicant State, and if the State applies as part of a consortium, all children in the consortium;
(7) Be developed consistent with universal design principles to be accessible to all children, including children with disabilities or developmental delays and English learners (as defined in this notice);
(8) As needed, provide appropriate accommodations and supports for children with disabilities or developmental delays and English learners (as defined in this notice) (e.g., augmentative communication devices and assistive technologies);
(9) Be administered soon enough after a child's enrollment into kindergarten to achieve the purposes for which the assessment was developed, including the purposes specified in paragraph (a) of this priority;
(10) Use multiple methods (e.g., performance tasks, selected responses, observational ratings) to measure children's performance and development;
(11) Be administered by a trained assessor or assessors;
(12) Be designed to incorporate technology in the collection of student data and in the process of assessing children's performance on learning and development tasks; and
(13) Be cost-effective to administer, maintain, and enhance during and after the project period.
(c) Technical Quality. The KEA must measure children's learning and development at kindergarten entry in ways that-
(1) Are consistent with nationally recognized professional and technical standards for assessment;
(2) Are consistent with the recommendations of the National Research Council report on early childhood assessments;6
Footnotes:
6 National Research Council. (2008). Early Childhood Assessment: Why, What, and How. Committee on Developmental Outcomes and Assessments for Young Children, C.E. Snow and S.B. Van Hemel, Editors. Board on Children, Youth, and Families, Board on Testing and Assessment, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. Available at www.nap.edu/catalog.php?record_id=12446.
(3) Are valid, reliable, and appropriate for their intended purposes;
(4) Provide a valid and reliable measure across the performance spectrum of each child's learning and development at kindergarten entry, including children with disabilities or developmental delays and English learners.
(d) Data. The KEA must produce data and information that-
(1) Allow, at kindergarten entry, for a valid and reliable interpretation of each child's learning and development across the essential domains of school readiness (as defined in this notice) with each domain making a significant contribution to the overall comprehensive score;
(2) Can be reported to and easily understood and used by various stakeholders, including families, teachers, administrators, early learning providers, and policy-makers, consistent with requirements of Federal, State, and local privacy laws; and
(3) Can be incorporated into a State's longitudinal data system (SLDS) and a State's early learning data system (if it is separate from an SLDS), consistent with requirements of Federal, State, and local privacy laws.
(e) Compatibility. The KEA must use approaches to assessment design and implementation (e.g., use of technology, assessment administration, scoring, and reporting) that facilitate the integration of the KEA with a State's student assessment system, including a State's comprehensive early learning assessment system (as defined in this notice) for each State included in an application in which a comprehensive early learning assessment system exists.
Proposed Priority 2-Early Learning Collaborative Efforts Among States
Background: The Department values the benefits derived from States working together and, therefore, proposes collaborative efforts among States as a priority for the development or enhancement of KEAs. As noted earlier, States are working together in consortia under the RTTA program to develop new assessment systems that measure student knowledge and skills against a common set of college- and career-ready standards in English language arts and mathematics. States are also collaborating under the GSEG program to develop companion alternate assessments based on alternate achievement standards. With assistance from the EAG program, States also are working together to develop ELP assessments aligned with common ELP standards.
Similarly, because of the complexity of developing or enhancing a KEA, States in collaboration may yield better results than those undertaking this effort alone. States working in collaboration can build on each State's expertise and experience and generate efficiencies in development, costs, implementation, and uses of results.
In addition, data produced by a KEA administered across multiple States are more meaningful when the early learning and development standards (as defined in this notice) are the same across States, and can provide a common framework for understanding the level of children's learning and development at kindergarten entry.
The Department is considering using this priority as a competitive preference priority in the FY 2013 competition. An applicant would receive a higher number of points based on the extent to which it includes a greater number of States in the consortium, with three to four States representing a low number of States, five to seven States representing an intermediate number of States, and eight or more States representing a high number of States.
Proposed Priority 2: Early Learning Collaborative Efforts Among States.
To meet this priority, an applicant must-
(a) Include a minimum of three States in the consortium and propose developing or enhancing a common KEA for those States. An applicant will receive a greater number of points under this priority based on the extent to which it includes a greater number of States in its consortium;
(b) Adopt or propose a plan for all States in the consortium to adopt a set of early learning and development standards (as defined in this notice) that, for at least the year prior to kindergarten entry, are substantially identical across all States in the consortium;
(c) Adopt or propose a plan for all States in the consortium to adopt the common KEA; and
(d) Provide in the memorandum of understanding or other binding agreement executed by each State in the consortium an assurance that, as a condition of remaining in the consortium, the State will, no later than the end of the project period, adopt the common KEA developed under this priority and the set of early learning and development standards (as defined in this notice) upon which the KEA is based.
Types of Priorities:
When inviting applications for a competition using one or more priorities, we designate the type of each priority as absolute, competitive preference, or invitational through a notice in the Federal Register . The effect of each type of priority follows:
Absolute priority: Under an absolute priority, the Department considers only applications that meet the priority (34 CFR 75.105(c)(3)).
Competitive preference priority: Under a competitive preference priority, we give competitive preference to an application by (1) awarding additional points, depending on the extent to which the application meets the priority (34 CFR 75.105(c)(2)(i)); or (2) selecting an application that meets the priority over an application of comparable merit that does not meet the priority (34 CFR 75.105(c)(2)(ii)).
Invitational priority: Under an invitational priority, we are particularly interested in applications that meet the priority. However, we do not give an application that meets the priority a preference over other applications (34 CFR 75.105(c)(1)).
Proposed Requirement:
Background: The proposed requirement is designed to support the transition to ongoing operational administration of assessments developed under the EAG program.
We would add this proposed requirement to the existing requirements for the EAG program established on April 19, 2011 (76 FR 21986). We list the existing requirements below to provide context and make commenting on the proposed requirement easier. We invite comment on the proposed requirement only. The existing requirements are that an eligible applicant awarded a grant under this program must:
(a) Evaluate the validity, reliability, and fairness of any assessments or other assessment-related instruments developed under a grant from this competition, and make available documentation of evaluations of technical quality through formal mechanisms (e.g., peer-reviewed journals) and informal mechanisms (e.g., newsletters), both in print and electronically;
(b) Actively participate in any applicable technical assistance activities conducted or facilitated by the Department or its designees, coordinate with the RTTA program in the development of assessments under this program, and participate in other activities as determined by the Department;
(c) Develop a strategy to make student-level data that result from any assessments or other assessment-related instruments developed under a grant from this competition available on an ongoing basis for research, including for prospective linking, validity, and program improvement studies;7
Footnotes:
7 Eligible applicants awarded a grant under this program must comply with the Family Educational Rights and Privacy Act (FERPA) and 34 CFR Part 99, as well as State and local requirements regarding privacy.
(d) Ensure that any assessments or other assessment-related instruments developed under a grant from this competition will be operational (ready for large-scale administration) at the end of the project period;
(e) Ensure that funds awarded under the EAG program are not used to support the development of standards, such as under the English language proficiency assessment system priority or any other priority;
(f) Maximize the interoperability of any assessments and other assessment-related instruments developed with funds from this competition across technology platforms and the ability for States to move their assessments from one technology platform to another by doing the following, as applicable, for any assessments developed with funds from this competition by-
(1) Developing all assessment items in accordance with an industry-recognized open-licensed interoperability standard that is approved by the Department during the grant period, without non-standard extensions or additions; and
(2) Producing all student-level data in a manner consistent with an industry-recognized open-licensed interoperability standard that is approved by the Department during the grant period;
(g) Unless otherwise protected by law or agreement as proprietary information, make any assessment content (i.e., assessments and assessment items) and other assessment-related instruments developed with funds from this competition freely available to States, technology platform providers, and others that request it for purposes of administering assessments, provided that those parties receiving assessment content comply with consortium or State requirements for test or item security; and
(h) For any assessments and other assessment-related instruments developed with funds from this competition, use technology to the maximum extent appropriate to develop, administer, and score the assessments and report results.
Proposed Requirement:
The Assistant Secretary proposes the following requirement for this program. The Department may apply this requirement in any year in which this program is in effect:
(i) Adopt and implement any assessments, other assessment-related instruments developed or enhanced under the proposed project, and any standards upon which they are based. In addition, if the applicant State applies as, or on behalf of a consortium of States, it must provide in any memorandum of understanding or other binding agreement executed by each State in the consortium an assurance that, to remain in the consortium, the State will adopt and implement any assessments or other assessment-related instruments developed or enhanced under the proposed project and any standards upon which they are based by the end of the project period.
Proposed Definitions:
Background:
Several important terms associated with the priorities, requirements, definitions, and selection criteria proposed in this notice are not defined in the EAG statute. We would add the proposed definitions to the existing definitions for the EAG program established on April 19, 2011 (76 FR 21986), though we are proposing to modify the definition of "English learner" established in 2011 in order to broaden the definition to include young children.
Proposed Definitions:
The Assistant Secretary proposes definitions for the EAG program. The Department may apply one or more of these new definitions, and any previously established definitions, in any year in which this program is in effect.
Comprehensive early learning assessment system means a coordinated and comprehensive system of multiple assessments, each of which is valid and reliable for its specified purpose and for the population with which it will be used, that organizes information about the process and context of young children's learning and development in order to help teachers make informed instructional and programmatic decisions and that conforms with the recommendations of the National Research Council report on early childhood assessments8by including, at a minimum: (a) Screening measures (as defined in this notice); (b) formative assessments; (c) measures of environmental quality (as defined in this notice); (d) measures of the quality of adult-child interactions (as defined in this notice); and (e) a kindergarten entry assessment (KEA).
Footnotes:
8 National Research Council (2008). Early Childhood Assessment: Why, What, and How. Committee on Developmental Outcomes and Assessments for Young Children, C.E. Snow and S.B. Van Hemel, Editors. Board on Children, Youth, and Families, Board on Testing and Assessment, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. Available at www.nap.edu/catalog.php?record_id=12446.
Early learning and development standards means a set of expectations, guidelines, or developmental milestones that-
(a) Describe what all children from birth to kindergarten entry should know and be able to do and their dispositions toward learning;
(b) Are appropriate for each age group (e.g., infants, toddlers, and preschoolers); for English learners; and for children with disabilities or developmental delays;
(c) Cover all essential domains of school readiness (as defined in this notice);
(d) Are universally designed and developmentally, culturally, and linguistically appropriate; and
(e) Are aligned with the State's K-3 academic standards in, at a minimum, early literacy and mathematics.
English learner means a child, including a child aged three and younger, who is an English learner consistent with the definition of a child who is "limited English proficient," as applicable, in section 9101(25) of the Elementary and Secondary Education Act of 1965, as amended.
Essential domains of school readiness means the domains of language and literacy development, cognition and general knowledge (including early mathematics and early scientific development), approaches toward learning, physical well-being and motor development (including adaptive skills), and social and emotional development.
Formative assessment (also known as a classroom-based or ongoing assessment) means assessment questions, tools, and processes-
(a) That are-
(1) Specifically designed to monitor children's progress;
(2) Valid and reliable for their intended purposes and their target populations; and
(3) Linked directly to the curriculum; and
(b) The results of which are used to guide and improve instructional practices.
Measures of environmental quality means valid and reliable indicators of the overall quality of the early learning environment.
Measures of the quality of adult-child interactions means the measures obtained through valid and reliable processes for observing how teachers and caregivers interact with children, where such processes are designed to promote child learning and to identify strengths and areas for improvement for early learning professionals.
Screening measures means age and developmentally appropriate, valid, and reliable instruments that are used to identify children who may need follow-up services to address developmental, learning, or health needs in, at a minimum, the areas of physical health, behavioral health, oral health, child development, vision, and hearing.
Proposed Selection Criteria:
Background: The Department intends that the selection criteria used for competitions for EAG funds will ensure that EAG projects address the most critical needs of education. The Department also expects that the selection criteria used for competitions for EAG funds will ensure that any assessments funded under this program will be of high technical quality. We established selection criteria for the EAG program on April 19, 2011 (76 FR 21986), and April 30, 2012 (77 FR 25470). The 2011 selection criteria addressed the assessment design and the assessment development plan; however, those criteria are not appropriate for entry assessments within a KEA. Therefore, we are proposing two new selection criteria that address similar issues but with a focus on kindergarten children.
The proposed selection criteria (h) and (i) would be used in combination with the selection criteria that have already been established. The Department notes that the 2011 assessment design selection criterion (b) is inconsistent with both the proposed kindergarten entry assessment design criterion (h) and the purposes of the proposed KEA priority, and the Department does not intend to use selection criterion (b) with the proposed KEA priority.
The Department also notes that the 2011 assessment development plan selection criterion (c) is inconsistent with both the proposed kindergarten entry assessment development plan selection criterion (i) and the purposes of the proposed KEA priority, and the Department does not intend to use the 2011 selection criterion (c) with the proposed KEA priority.
We list the existing selection criteria below to provide context and to make commenting on the proposed selection criteria easier. We invite comments on the proposed selection criteria only.
The existing selection criteria are:
(a) Theory of action. The Secretary reviews each application to determine the extent to which the eligible applicant's theory of action is logical, coherent, and credible, and will result in improved student outcomes. In determining the extent to which the theory of action has these attributes, we will consider the description of, and rationale for-
(1) How the assessment results will be used (e.g., at the State, local educational agency, school, classroom, and student levels);
(2) How the assessments and assessment results will be incorporated into coherent educational systems (i.e., systems that include standards, assessments, curriculum, instruction, and professional development) of the State(s) participating in the grant; and
(3) How those educational systems as a whole will improve student achievement.
(b) Assessment design. The Secretary reviews each application to determine the extent to which the design of the eligible applicant's proposed assessments is innovative, feasible, and consistent with the theory of action. In determining the extent to which the design has these attributes, we will consider-
(1) The number and types of assessments, as appropriate (e.g., diagnostic assessments, summative assessments);
(2) How the assessments will measure student knowledge and skills against the full range of the relevant standards, including the standards against which student achievement has traditionally been difficult to measure, provide an accurate measure of student proficiency on those standards, including for students who are high- and low-performing in academic areas, and provide an accurate measure of student progress in the relevant area over a full academic year;
(3) How the assessments will produce the required student performance data, as described in the priority;
(4) How and when during the academic year different types of student data will be available to inform and guide instruction, interventions, and professional development;
(5) The types of data that will be produced by the assessments, which must include student achievement data and other data specified in the relevant priority;
(6) The uses of the data that will be produced by the assessments, including (but not limited to)-
(i) Determining individual student achievement and student progress; determining, as appropriate and as one of multiple measures, individual principal and teacher effectiveness, if applicable; and professional development and support needs;
(ii) Informing teaching, learning, and program improvement; and
(7) The frequency and timing of administration of the assessments, and the rationale for these;
(8) The number and types of items (e.g., performance tasks, selected responses, observational rating, brief or extended constructed responses) and the distribution of item types within the assessments, including the extent to which the items will be varied and elicit complex student demonstrations or applications of knowledge, skills, and approaches to learning, as appropriate (descriptions should include a concrete example of each item type proposed); and the rationale for using these item types and their distributions;
(9) The assessments' administration mode (e.g., paper-and-pencil, teacher rating, computer-based, or other electronic device), and the rationale for the mode;
(10) The methods for scoring student performance on the assessments, the estimated turnaround times for scoring, and the rationale for these; and
(11) The reports that will be produced based on the assessments, and for each report: the key data it will present; its intended use; target audience (e.g., students, parents, teachers, administrators, policymakers); and its presentation in an understandable and uniform format and, to the extent practicable, in a language that parents can understand.
(c) Assessment development plan. The Secretary reviews each application to determine the extent to which the eligible applicant's plan for developing the proposed assessments will ensure that the assessments are ready by the end of the grant period for wide-scale administration in a manner that is timely, cost-effective, and consistent with the proposed design and incorporates a process for ongoing feedback and improvement. In determining the extent to which the assessment development plan has these attributes, the Department will consider-
(1)(i) The approaches for developing assessment items (e.g., evidence-centered design, universal design) and the rationale for using those approaches; and the development phases and processes to be implemented consistent with the approaches; and
(ii) The types of personnel (e.g., practitioners, content experts, assessment experts, experts in assessing English learners, linguists, experts in second language acquisition, experts in assessing students with disabilities, psychometricians, cognitive scientists, institution of higher education representatives, experts on career readiness standards, and other key stakeholders) involved in each development phase and process;
(2) The approach and strategy for designing and developing accommodations, accommodation policies, and methods for standardizing the use of those accommodations for students with disabilities;
(3) The approach and strategy for ensuring scalable, accurate, and consistent scoring of items, including the approach and moderation system for any human-scored items and the extent to which teachers are trained and involved in the administration and scoring of assessments;
(4) The approach and strategy for developing the reporting system; and
(5) The overall approach to quality control and the strategy for field-testing assessment items, accommodations, scoring systems, and reporting systems, including, with respect to assessment items and accommodations, the use of representative sampling of all types of student populations, taking into particular account high- and low-performing students, different types of English learners (e.g., recently arrived English learners, former English learners, migratory English learners, and English learners with disabilities), and students with disabilities.
(d) Research and evaluation. The Secretary reviews each application to determine the extent to which the eligible applicant's research and evaluation plan will ensure that the assessments developed are valid, reliable, and fair for their intended purposes. In determining the extent to which the research and evaluation plan has these attributes, we will consider-
(1) The plan for identifying and employing psychometric techniques suitable for verifying, as appropriate to each assessment, its construct, consequential, and predictive validity; external validity; reliability; fairness; precision across the full performance continuum; and comparability within and across grade levels; and
(2) The plan for determining whether the assessments are being implemented as designed and the theory of action is being realized, including whether the intended effects on individuals and institutions are being achieved.
(e) Professional capacity and outreach. The Secretary reviews each application to determine the extent to which the eligible applicant's plan for implementing the proposed assessments is feasible, cost-effective, and consistent with the theory of action. In determining the extent to which the implementation plan has these attributes, we will consider-
(1) The plan for supporting teachers and administrators in implementing the assessments and for developing, in an ongoing manner, their professional capacity to use the assessments and results to inform and improve instructional practice; and
(2) The strategy and plan for informing the public and key stakeholders (including teachers, administrators, families, legislators, and policymakers) in each State or in each member State within a consortium about the assessments and for building support from the public and those stakeholders.
(f) Technology approach. The Secretary reviews each application to determine the extent to which the eligible applicant would use technology effectively to improve the quality, accessibility, cost-effectiveness, and efficiency of the proposed assessments. In determining the extent to which the eligible applicant is using technology effectively, we will consider-
(1) The description of, and rationale for, the ways in which technology will be used in assessment design, development, administration, scoring, and reporting; the types of technology to be used (including whether the technology is existing and commercially available or is being newly developed); and how other States or organizations can re-use in a cost-effective manner any technology platforms and technology components developed under this grant; and
(2) How technology-related implementation or deployment barriers will be addressed (e.g., issues relating to local access to internet-based assessments).
(g) Project management. The Secretary reviews each application to determine the extent to which the eligible applicant's project management plan will result in implementation of the proposed assessments on time, within budget, and in a manner that is financially sustainable over time. In determining the extent to which the project management plan has these attributes, we will consider-
(1) The project workplan and timeline, including, for each key deliverable (e.g., necessary procurements and any needed approvals for human subjects research, assessment, scoring and moderation system, professional development activities), the major milestones, deadlines, and entities responsible for execution;
(2) The approach to identifying, managing, and mitigating risks associated with the project;
(3) The extent to which the eligible applicant's budget is adequate to support the development of assessments that meet the requirements of the priority and includes costs that are reasonable in relation to the objectives, design, and significance of the proposed project and the number of students to be served;
(4) For each applicant State or for each member State within a consortium, the estimated costs for the ongoing administration, maintenance, and enhancement of the operational assessments after the end of the project period for the grant and a plan for how the State will fund the assessments over time (including by allocating to the assessments funds for existing State or local assessments that will be replaced by the new assessments); and
(5) The quality and commitment of the personnel who will carry out the proposed project, including the qualifications, relevant training, and experience of the project director and other key project personnel, and the extent to which the time commitments of the project director and other key project personnel are appropriate and adequate to meet the objectives of the proposed project.
Proposed Selection Criteria:
The Assistant Secretary proposes the following selection criteria for evaluating an application under this program. We may apply these criteria or any of the existing selection criteria in any year in which this program is in effect. In the notice inviting applications and the application package, the Department will announce the selection criteria to be applied and the maximum possible points assigned to each criterion.
(h) Kindergarten entry assessment design.
The Secretary reviews each application to determine the extent to which the design of the eligible applicant's proposed assessment is innovative, feasible, and consistent with the theory of action. In determining the extent to which the design has these attributes, the Department will consider-
(1) How the assessment will measure child performance and development against early learning and development standards (as defined in this notice);
(2) The steps proposed for ensuring that the assessment is aligned with the specific early learning and development standards on which the assessment is based;
(3) The extent to which data from the assessment can be incorporated into a State's longitudinal data system (SLDS) and a State's early learning data system (if it is separate from an SLDS) through the use of or connection to common data elements and definitions, such as the Common Education Data Standards ( https://ceds.ed.gov/ ), consistent with requirements of Federal, State, and local privacy laws;
(4) The intended uses of the data to be generated by the assessment, which must include, but need not be limited to-
(i) Determining the level of individual child learning and development;
(ii) Identifying teacher professional development and support needs;
(iii) Informing teaching, learning, and program improvement; and
(iv) Engaging families in the early learning of their children;
(5) The number and types of items (e.g., performance tasks, selected responses, observational ratings) and the distribution of item types within the assessment, including the variation of the items and the rationale for using these item types and their distributions;
(6) The assessment's administration mode(s) (e.g., direct, observation, or administered using an electronic device), and the rationale for the mode(s);
(7) The methods for scoring child performance on the assessments, the estimated turnaround times for scoring, and the rationale(s) for these;
(8) The applicant's plan to set levels of performance for the assessment, where the levels of performance encompass descriptors of what a child knows and is able to do for each level, and for how the applicant will meaningfully engage and solicit stakeholder input on the development of levels of performance that are valid and reliable for children's learning and development; and
(9) The reports and interpretation guides that will be produced based on the assessments, and for each report and interpretation guide: the key data it will present; its intended use; its target audience (e.g., families, teachers, administrators, policymakers, and other stakeholders); and how its presentation will be in an understandable and uniform format and, to the extent practicable, in a language that families can understand.
(i) Kindergarten entry assessment development plan. The Secretary reviews each application to determine the extent to which the eligible applicant's plan for developing the proposed KEA will ensure that the assessments are ready by the end of the grant period for wide-scale administration in a manner that is timely, cost-effective, and consistent with the proposed design and incorporates a process for ongoing feedback and improvement. In determining the extent to which the assessment development plan has these attributes, the Department will consider-
(1)(i) The approaches for developing assessment items (e.g., evidence-centered design, universal design), the rationale for using those approaches, and the development phases and processes to be implemented consistent with the approaches;
(ii) The types of personnel involved in each development phase and process (e.g., practitioners, experts in early learning and development, experts in the assessment of young children, content experts, assessment experts, experts in assessing children with disabilities or developmental delays and English learners, psychometricians, cognitive scientists, and other key stakeholders);
(2) The approach and strategy for designing and developing accommodations, accommodation policies, and methods for standardizing the use of those accommodations for children with disabilities or developmental delays and English learners (as defined in this notice);
(3) The approach and strategy for ensuring scalable, accurate, and consistent scoring of items, including the approach and moderation system for any items not scored by machine and the extent to which teachers are trained and involved in the administration and scoring of assessments;
(4) The approach and strategy for developing the reporting system; and
(5) The overall approach to quality control, maintaining the integrity of the assessment process, field-testing assessment items, accommodations, scoring systems, and reporting systems, including, with respect to assessment items and accommodations, the use of representative sampling of all types of child populations, taking into particular account the full range of learning and development across the essential domains of school readiness (as defined in this notice), and including children with disabilities or developmental delays and English learners (as defined in this notice).
Final Priorities, Requirements, Definitions, and Selection Criteria:
We will announce the final priorities, requirements, definitions, and selection criteria in a notice in the Federal Register . We will determine the final priorities, requirements, definitions, and selection criteria after considering responses to this notice and other information available to the Department. This notice does not preclude us from proposing additional priorities, requirements, definitions, or selection criteria, subject to meeting applicable rulemaking requirements.
Note:
This notice does not solicit applications. In any year in which the Department chooses to use these priorities, requirements, definitions, or selection criteria, we invite applications through a notice in the Federal Register .
Executive Order 12866 and 13563
Regulatory Impact Analysis
Under Executive Order 12866, the Secretary must determine whether this regulatory action is "significant" and, therefore, subject to the requirements of the Executive Order and subject to review by the Office of Management and Budget (OMB). Section 3(f) of Executive Order 12866 defines a "significant regulatory action" as an action likely to result in a rule that may-
(1) Have an annual effect on the economy of $100 million or more, or adversely affect a sector of the economy, productivity, competition, jobs, the environment, public health or safety, or State, local or tribal governments or communities in a material way (also referred to as an "economically significant" rule);
(2) Create serious inconsistency or otherwise interfere with an action taken or planned by another agency;
(3) Materially alter the budgetary impacts of entitlement grants, user fees, or loan programs or the rights and obligations of recipients thereof; or
(4) Raise novel legal or policy issues arising out of legal mandates, the President's priorities, or the principles set forth in the Executive Order.
This proposed regulatory action is not a significant regulatory action subject to review by OMB under section 3(f) of Executive Order 12866.
We have also reviewed this proposed regulatory action under Executive Order 13563, which supplements and explicitly reaffirms the principles, structures, and definitions governing regulatory review established in Executive Order 12866. To the extent permitted by law, Executive Order 13563 requires that an agency-
(1) Propose or adopt regulations only upon a reasoned determination that their benefits justify their costs (recognizing that some benefits and costs are difficult to quantify);
(2) Tailor its regulations to impose the least burden on society, consistent with obtaining regulatory objectives and taking into account-among other things and to the extent practicable-the costs of cumulative regulations;
(3) In choosing among alternative regulatory approaches, select those approaches that maximize net benefits (including potential economic, environmental, public health and safety, and other advantages; distributive impacts; and equity);
(4) To the extent feasible, specify performance objectives, rather than the behavior or manner of compliance a regulated entity must adopt; and
(5) Identify and assess available alternatives to direct regulation, including economic incentives-such as user fees or marketable permits-to encourage the desired behavior, or provide information that enables the public to make choices.
Executive Order 13563 also requires an agency "to use the best available techniques to quantify anticipated present and future benefits and costs as accurately as possible." The Office of Information and Regulatory Affairs of OMB has emphasized that these techniques may include "identifying changing future compliance costs that might result from technological innovation or anticipated behavioral changes."
We are issuing these proposed priorities, requirements, definitions, and selection criteria only on a reasoned determination that their benefits would justify their costs. In choosing among alternative regulatory approaches, the Department selected those approaches that would maximize net benefits. Based on the analysis that follows, the Department believes that this regulatory action is consistent with the principles in Executive Order 13563.
We also have determined that this regulatory action would not unduly interfere with State, local, and tribal governments in the exercise of their governmental functions.
In accordance with both Executive orders, the Department has assessed the potential costs and benefits, both quantitative and qualitative, of this regulatory action. The potential costs are those resulting from statutory requirements and those we have determined as necessary for administering the Department's programs and activities.
The proposed priority for KEAs and the other proposed priority, along with the associated proposed requirement, definitions, and selection criteria, would benefit individual children by supporting the development or enhancement of KEAs that would provide educators with timely and useful information to guide individualized instruction for children at kindergarten entry and throughout the year. In addition, the resulting assessments would benefit educators, administrators, and other stakeholders by yielding information that can be used to target investments for the education systems serving children in the years before kindergarten. A KEA would also support the implementation of State reform efforts in the area of early learning.
The proposed priority for early learning collaborative efforts among States would encourage States to work together on developing a common KEA rather than developing or using separate KEAs, thus pooling expertise and experience while also creating efficiencies, including cost-efficiencies. The priority would also help ensure that a KEA developed by a consortium is made available for use by multiple States. It also would support the collection of comparable data regarding the level of children's learning and development at kindergarten entry.
The proposed selection criteria would help ensure that the assessments developed by grantees are of high quality, meet relevant technical standards, and align with other assessment work funded by the Department.
Intergovernmental Review: This program is subject to Executive Order 12372 and the regulations in 34 CFR part 79. One of the objectives of the Executive Order is to foster an intergovernmental partnership and a strengthened federalism. The Executive Order relies on processes developed by State and local governments for coordination and review of proposed Federal financial assistance.
This document provides early notification of our specific plans regarding this program.
Accessible Format: Individuals with disabilities can obtain this document in an accessible format (e.g., braille, large print, audiotape, or compact disc) on request to the program contact person listed under FOR FURTHER INFORMATION CONTACT .
Electronic Access to This Document: The official version of this document is the document published in the Federal Register . Free Internet access to the official edition of the Federal Register and the Code of Federal Regulations is available via the Federal Digital System at: www.gpo.gov/fdsys. At this site you can view this document, as well as all other documents of this Department published in the Federal Register , in text or Adobe Portable Document Format (PDF). To use PDF you must have Adobe Acrobat Reader, which is available free at the site.
You may also access documents of the Department published in the Federal Register by using the article search feature at: www.federalregister.gov. Specifically, through the advanced search feature at this site, you can limit your search to documents published by the Department.
Dated: January 22, 2013.
Deborah S. Delisle,
Assistant Secretary for Elementary and Secondary Education.
[FR Doc. 2013-01567 Filed 1-24-13; 8:45 am]
BILLING CODE 4000-01-P