AI Rec 12 Evaluation

States should begin developing evaluation frameworks to determine the impact of AI on K-12 and postsecondary education.

To calculate AI’s impact on education, states will need to determine the questions they need to answer and then identify the data they will need collect to answer those questions. 

Where to Start?

• Option 1: Each state develops their own evaluation framework.
• Option 2: The 16 SREB states could work together through the upcoming AI Advisory panel to develop a regional evaluation framework. Once developed, the framework could be piloted, revised and shared across the states. This would save time and effort while providing comparable data on AI impact across the SREB states.

Starting the Process

The following sample questions and indicators have been developed to help states interested in evaluating AI’s impact on education begin the process of developing evaluation frameworks.

Overall Impact

• Is AI use aligned to our educational goals?
• Is AI helping us achieve our educational goals and objectives?
  • If so, how and why?
  • If not, why?
• What challenges are we encountering?
• How well is AI working in K-12?
• How well is AI working in postsecondary education?
• What is the magnitude of AI’s impact?
• Are there any policy changes that would support better use of AI in education?
• Are there examples of best practices related to AI use in education?
• What are the lessons learned regarding AI use to date?

Students and AI

• Does AI improve student motivation, attendance or persistence in learning tasks?
• Are students receiving more tailored instruction and support?
  • How measurable is the improvement?
• Does AI support the development of critical thinking, creativity or AI literacy itself?
• Is AI helping the outcomes for subgroups?
• Do AI-enhanced programs better prepare students for postsecondary education or careers?
• Do students feel empowered or surveilled by AI tools?
• Do families understand and support how AI is used with their children’s data?
• How does AI affect student academic performance and concept mastery?

Teaching and AI

• Does AI save teachers time grading, lesson planning or administrative work?
• How do AI tools change teaching practices — are they enhancing or narrowing pedagogy?
• Are teachers receiving effective training to integrate AI responsibly?
• Do educators feel AI supports their expertise, or undermines their professional judgment?

Operations & Efficiency

• Are AI tools delivering measurable value compared to their cost?
• How much time and money is saved in scheduling, HR, transportation or facilities?
• Are leaders using AI insights to make better, data-driven decisions?
• Can pilot projects scale across classrooms, schools or districts without loss of quality?

Policy, Ethics & Governance

• Are AI tools compliant with FERPA, COPPA and state-level student data protections?
• Bias & Fairness: Are outputs similar or do they replicate systemic biases?
• Do stakeholders such as teachers, students and parents understand how AI makes decisions?
• Who is responsible when AI systems fail, make errors or cause harm?
• Are local employers, postsecondary institutions, and communities seeing benefits from AI-enhanced education (such as workforce readiness or alignment with “Profile of a Graduate” skills)?
• Are AI programs resilient beyond initial grant or ESSER funding?
• Is AI reshaping governance, resource allocation or instructional models long-term?

We advise connecting these questions to measurable indicators (assessment results, teacher time saved, cost-per-student, incident reports on bias, survey responses) and align them with policy goals (access, accountability or innovation).

AI in Education Impact Measurement Framework

Student Learning & Outcomes

How is AI affecting student achievement, engagement and skills development? Are gains consistent across subgroups?

Indicators
Academic performance (state test scores, course grades, formative assessment data)
Engagement (attendance, LMS logins, time-on-task analytics)
Skill growth (critical thinking, AI literacy, creativity, career-ready competencies)
Gaps (outcome differences across demographic groups)

Data Sources
State and district assessments
LMS/AI tool usage logs
Student surveys and focus groups
Longitudinal data systems (P-20W)
Teachers (classroom-level measures)
District research & accountability teams
State education agencies

Teaching & Professional Practice

Does AI save teachers time and improve instructional quality? Do educators feel supported, not displaced?

Indicators
Hours saved per week (grading, planning, admin tasks)
Teacher satisfaction & trust in AI (survey ratings)
PD completion rates & demonstrated AI integration skills
Classroom observation rubrics (pedagogy shifts)

Data Sources
Teacher time logs and self-reports
PD participation records
Classroom walkthroughs and evaluations
Teacher focus groups
Responsible Parties
School principals
District HR & professional development teams
Teacher unions or professional associations

System-Level Operations & Efficiency

Is AI improving efficiency in operations and decision-making? Do cost savings outweigh investments?

Indicators
Cost per student served by AI tools
Reduction in admin hours (HR, transportation, sceduling)
Predictive accuracy of AI analytics (attendance, risk, transportation)
Scalability measures (percentage of schools implementing AI solutions)

Data Sources
District finance and budget reports
Operational system data (transportation, HR, facilities)
Procurement and RFP evaluations
Responsible Parties
District operations leaders
CFO/finance officers
State auditors

Policy, Ethics & Guidance

Are AI uses compliant with laws, ethical guidelines, and governance frameworks? Who is accountable for misuse or harm?

Indicators
FERPA or COPPA compliance audit results
Data breach or misuse incident reports
Vendor transparency (model documentation, explainability)
Number of board policies updated for AI governance

Data Sources
Legal and policy compliance reviews
Incident tracking system
Vendor disclosures or contracts
School board policy manuals
Responsible Parties
District general counsel
State regulators
School boards

Student & Community Voice

Do students and parents feel AI empowers or surveils them? Are communities seeing broader benefits?

Indicators
Student trust in AI (survey items, interviews)
Parent awareness & support (focus groups, surveys)
Community partnerships (employer/postsecondary engagement with AI programs)
Complaints or advocacy activity (school board meetings, public forums)

Data Sources
Annual stakeholder surveys
Public comment records
Community partnership agreements
Student advisory groups
Responsible Parties
PTA/family engagement coordinators
Community college or workforce boards
District superintendents

Longitudinal Impact

Are AI impacts sustainable and systemic over time? Do they improve readiness for careers and college?

Indicators
Post-graduation outcomes (college enrollment, workforce placement, credential attainment)
Long-term cost-effectiveness (ROI across 3–5 years)
AI integration into strategic plans or profiles of a graduate
Sustainability of funding (state support vs. short-term ESSER/Title grants)

Data Sources
State longitudinal data systems (P-20W)
Postsecondary enrollment and workforce data
Multi-year budget reports
Strategic plans and state accountability reports
Responsible Parties
State education agencies
Workforce boards
Governors’ offices or state legislatures

References

American Institute for Research. (2024). The impact of artificial intelligence on higher education: An empirical study. European Journal of Educational Sciences, 11(1). https://eric.ed.gov/?id=EJ1384682

Chan, C. Y., & Zhou, W. (2023). Deconstructing student perceptions of generative AI (GenAI) through an Expectancy-Value Theory (EVT)-based instrument. arXiv. https://arxiv.org/abs/2305.01186 arXiv

Dwivedi, Y. K., et al. (2023). Ethical principles for artificial intelligence in education. Education and Information Technologies, 28, 1-22. https://doi.org/10.1007/s10639-022-11316-w

Ghimire, A., Prather, J., & Edwards, J. (2024). Generative AI in education: A study of educators’ awareness, sentiments, and influencing factors. arXiv. https://arxiv.org/abs/2403.15586 arXiv

Kanellopoulos, S., et al. (2024). Artificial intelligence in education: A systematic literature review. Expert Systems with Applications, 240, 116561. https://doi.org/10.1016/j.eswa.2023.116561

Liang, Y., & Miao, G. (2023). AI literacy in K-12: A systematic literature review. International Journal of STEM Education,8(1). https://doi.org/10.1186/s40594-023-00418-7

Lin, X., et al. (2024). The impact of artificial intelligence (AI) on students’ learning processes and academic performance. Education Sciences,15(3), 343. https://doi.org/10.3390/educsci15030043

Mao, Y., Nguyen, T. N., & Sarrion, G. (2024). Artificial intelligence in educational leadership: A comprehensive taxonomy of AI applications. Journal of Educational Technology & Society, 27(1), 123-145. https://doi.org/10.1186/s41239-025-00517-1

McGehee, N. (2024, June 21). AI in education: Student usage in online learning. Michigan Virtual Learning Research Institute. https://michiganvirtual.org/

OECD. (2024, August). The potential impact of artificial intelligence on equity and inclusion in education. OECD Publishing. https://doi.org/10.1787/15df715b-en OECD

Panigrahi, R., et al. (2023). Practical and ethical challenges of large language models in education: A systematic scoping review. arXiv. https://arxiv.org/abs/2303.13379 arXiv

Saad, M., & Tounkara, F. (2025). A systematic review of the early impact of artificial intelligence on education. Frontiers in Education, 10, 1522841. https://doi.org/10.3389/feduc.2025.152284

Sergeeva, O. V., Zheltukhina, M. R., Shoustikova, T., Tukhvatullina, L. R., Dobrokhotov, D. A., & Kondrashev, S. V. (2025). Understanding higher education students’ adoption of generative AI technologies: An empirical investigation using

UTAUT2. Contemporary Educational Technology, 17(2), ep571. https://doi.org/10.30935/cedtech/16039

U. S. Department of Education, Office of Educational Technology. (2023). Artificial intelligence and the future of teaching and learning: Insights and recommendations (EDI-AI Report). U.S. Department of Education. https://tech.ed.gov/ai-report/

Wagner, C., et al. (2024). Embracing the future of Artificial Intelligence in the classroom. Journal of Educational Technology & Society, 27(1), 123–145. https://doi.org/10.1186/s41239-024-00448-3

Wang, C., et al. (2024). Exploring the impact of artificial intelligence on higher education in Saudi Arabia. Humanities and Social Sciences Communications, 11, 82. https://doi.org/10.1057/s41599-024-03432-4

West, D. M., & Allen, J. (2024). A review of artificial intelligence in K-12 education. Education Research International. (2023). https://doi.org/10.1155/2023/135198

World Bank. (2024). AI Revolution in Education (Brief No. 1). World Bank. https://documents1.worldbank.org/curated/en/099734306182493324/pdf/IDU152823b13109c514ebd19c241a289470b6902.pdf World Bank

Yan, L., Sha, L., Zhao, L., Li, Y., Martínez-Maldonado, R., Chen, G., Li, X., & Jin, Y. (2023). Practical and ethical challenges of large language models in education: A systematic scoping review. arXiv. https://arxiv.org/abs/2303.13379 arXiv

Zawacki-Richter, O., et al. (2025). A meta-systematic review of artificial intelligence in higher education (AIHEd). Journal of Educational Technology & Society. https://doi.org/10.1186/s41239-023-00436-z

Zhou, H., Wang, Y., & Zhang, W. (2024). Artificial intelligence in education: A systematic literature review. Expert Systems with Applications, 240, 116561. https://doi.org/10.1016/j.eswa.2023.116561