Time to End the Keystone Exam Charade

(September 21, 2023)–Background:  In 2010 the Pennsylvania Legislature passed a law establishing an end-of-course assessment system known as the Keystone Exams.  They were to be administered to all 11^th grade students covering Algebra, Literature (English) and Biology to establish high-school graduation requirements and provide a way to hold high schools accountable for educating students. However, the assessment system has been a great failure in terms of meeting its objectives despite huge costs to taxpayers.
____________________________________________________________________

Overview and history

As noted in Policy Brief, Vol. 22, No. 13, students began taking the exams in 2012. Scoring proficient on the exams was to become a graduation requirement in 2017.  However, in the 2014-15 school year (the graduating class of 2016) the results were a great disappointment.  Statewide 123,091 students took the Algebra portion of the exam and more than a third of the students tested (35.5 percent) scored basic or below basic—that means they were less than the proficient level necessary to meet graduation requirements.  For the English portion 122,568 students took the exam and 27 percent scored basic or below basic.

In response the Legislature in 2016 passed Act 1 to delay using the Keystone Exams as a graduation requirement.  As each year passed the results did not improve and the Legislature kept pushing back the requirement.  In 2020, during the pandemic, the exam was not given.  The exams are administered three times per year and any student that does not score proficient or above is allowed to retake the exam until they do so.  If a student doesn’t score proficient or above, they may be provided supplemental instruction consistent with the student’s educational program by the student’s district.

The table below shows the statewide results from 2017 through 2022 for the exams taken by all students statewide.

The results show a consistency in the percentage of students scoring basic or below basic for the three modules.  The exception is the uptick in the percentage of students scoring basic or below basic on the Literature exam in 2021 and 2022.

According to the Pennsylvania Public School Code, “…the use of the Keystone Exams as a graduation requirement or as a benchmark for the need for participation in a project-based assessment shall be delayed until the 2022-2023 school year.”  But as that previous Brief noted, in 2018 Act 158 was passed providing over 20 alternative accomplishments that can be used to meet graduation requirements for students who cannot reach proficiency levels on the exams.

Among the alternatives are:

1. successful completion of locally established, grade-based requirements for academic content areas associated with each Keystone Exam on which the student did not achieve at least a proficient score on an approved alternative assessment.
2. successful completion of a concurrent enrollment course in an academic content area associated with each Keystone Exam on which the student did not achieve at least a proficient score.
3. acceptance to an accredited four-year nonprofit institution of higher education and evidence of the ability to enroll in college-level, credit-bearing coursework.
4. successful completion of a pre-apprenticeship program.
5. a letter guaranteeing full-time employment.

As a result of these alternatives, the number of test takers has declined.  The following table shows those totals from 2017 through 2022.

The number of test takers was consistent from 2017 through 2019, although 2019 did see a modest drop.  After the alternatives to the test were put into place post-pandemic, the number of test takers predictably began to decline. From 2019 to 2021, the number of test takers in Algebra fell by 15 percent and a remarkable 91 percent in Literature and 52 percent in Biology.  There was a rebound in test takers for Literature, but it was still 13.5 percent lower than in 2019.  The number of test takers in Algebra fell further dipping 42 percent from 2021 to 2022 and was 50.5 percent lower than in 2019.  Meanwhile, Biology test takers are still 47 percent lower than the pre-pandemic level.

Notwithstanding the large decline in test takers, test scores are not much improved.

Allegheny County high schools

To gauge the impact of Act 158 of 2018 and the addition of Keystone Exam alternatives, test results for high schools in Allegheny County from both 2019 and 2022 are examined.

In Allegheny County in 2019, there were 51 public high schools in 41 school districts.  The Pittsburgh Public School District has 11 high schools while neither the Duquesne nor Wilkinsburg School Districts have a high school.  There were also 15 charter or technical high schools.  Combined they administered the Keystone Exams to 9,893 students in Algebra and 9,844 students in Literature (8 percent of the state total).

On the Algebra exam, only 3 school district high schools had 90 percent or more of test takers score at the proficient or advanced level.  Another 10 had overall scores between 80 and 90 percent.  The top performing Pittsburgh Public high school was the Science and Technology Academy with 79.1 percent of test takers scoring proficient or advanced.  The average percentage of students scoring proficient or advanced among the 60 high schools reporting scores was 56.4 percent (five charter and one public high school did not report results for Algebra). Put another way, on average 43.6 percent of Allegheny high school test takers scored basic or below basic.

On the Literature exam, results were a little better.

Nine public high schools had 90 percent or more of test takers score advanced or proficient.  Another 16 schools had between 80 and 90 percent of test takers score proficient or advanced.  The average percentage of 11^th grade students scoring advanced or proficient among all high schools in Allegheny County was 67.2 percent.  Seven schools (six charter and one public high school) did not report results for the Literature exam. Looked at the other way, 32.8 percent of test takers scored basic or below on Literature—less than the state average of 35.1 percent.

With the alternatives to taking the exam in place, fewer Allegheny County high school students opted to take the exams in 2022. The score results were mixed.

On the Algebra module, the number of students taking the exam fell to 5,557, a decline of 43.8 percent from 2019.  It is notable that the number of charter and technical schools with test takers fell to 13.  The number of high schools having 90 percent or more test takers scoring proficient or advanced rose to nine with another nine schools reporting scores between 80 and 90 percent.  The 18 schools with 80 percent or higher scoring proficient or advanced jumped from just 13 in 2019.  Also, for 2022, 15 schools (six public and nine charter or technical schools) did not report results, up from 2019’s total of six.

The average percentage proficient or advanced increased to 66.5 percent. As might have been predicted, as the number of test takers fell scores improved because most test takers opting to take the test would have been fairly confident they could do well.

The results of the Literature exam were not as good.

The number of test takers fell to 8,561, a drop of 13 percent from 2019—a much smaller decline than for Algebra. Unlike the Algebra results, the number of high schools that had test takers scoring 90 percent proficient or advanced fell from nine to five.  The number of schools that had test takers scoring between 80 and 90 percent fell from 16 in 2019 to 7 in 2022.  The average percentage of students scoring proficient or advanced fell from 67.2 percent to 60.9 percent.  While one year doesn’t make a trend, that type of fall is worrisome.  Perhaps more students opted out of Algebra out of a fear of math and yet overestimated their Literature skills?

Conclusion

Oddly enough, from one perspective the Keystone Exams assessment system is not a total failure.  The exams revealed an all too predictable truth based on years of evidence regarding education performance across the state. Despite the enormous sums of money the state and local districts spend on education, the overall level of academic achievement is, and has been, quite disappointing.

The passing of Act 158 of 2018, which provides 20 alternatives other than taking the Keystone Exams is an admission that the overall level of academic performance is inadequate.  The Commonwealth spent millions of dollars over the last decade or more creating and administering the Keystone Exams.  It’s time to end the charade and eliminate them altogether and rethink how to improve education in Pennsylvania.  Unfortunately, past experience regarding education improvement in far too many schools is not encouraging.

Frank Gamrat, Ph.D., Executive Director

If you wish to support our efforts please consider becoming a donor to the Allegheny Institute.The Allegheny Institute is a 501(c)(3) non-profit organization and all contributions are tax deductible.Please mail your contribution to:

The Allegheny Institute
305 Mt. Lebanon Boulevard
Suite 208
Pittsburgh, PA15234

For more on this topic click here.

Link to Allegheny Institute Website

To read our thrice-weekly “At-Large” columns, click here.