The New State Report Card proposal.

In the waiver request filed with the federal government to excuse the state from the provisions of No Child Left Behind legislation, the state provided the roadmap for its current thinking with regard to the state report card. What the state of Ohio has decided to do, at least in the short term, is to maintain most of the existing metrics but simply grade downward in an effort to give less "Excellent" ratings on the report card. Whether their motivation was to combat grade inflation or to make their waiver request more appealing to the federal government, a new report card is coming to Ohio.

The new report card would be made up of 4 components and they are all graded separately and given equal weight in determining the final grade. The components are:

1) The scores on the proficiencies

2) The performance index

3) The value-add assessment

4) A new metric designed to measure the success a district has at eliminating the achievement gap

The overall grade is determined by averaging the grades on the 4 metrics.

Let's look at each in turn.

PROFICIENCIES

At least through 2015, the proficiency tests will remain largely as-is, cut scores and all,  so, for example, a district will continue to receive credit for passing the 6th grade reading proficiency if 75% of the students get a 35% on the test. If you pass 90% of the available proficiencies, your district receive an "A" on that particular metric with 80% required to get a "B".

PERFORMANCE INDEX

The performance index metric attempts to give the district credit for those students scoring in the "advanced" and "accelerated" range on the tests, but as we see from the charts in the previous posts, the cut off scores for advanced and accelerated remain at the same low levels that they were before, therefore, the performance index metric should be similarly unchanged. The difference is in the score required to receive an "A". If you receive a performance index of 90% of the maximum, or 108 out of a possible 120 points, you get your "A".  Only one school in Worthington, Evening Street Elementary, received a performance index in excess of 108. It's score was 108.8, however, we must remember that the performance index is only concerned with how many students clear the proficient, accelerated and advanced bar and not necessarily by how much those students passed the bar. Let's take an example from a school that would narrowly miss getting that magic 108 to receive the "A" on the performance index component.

Performance Index Calculation for Phoenix Middle School, Worthington OH

The example demonstrates that even with 95% of the scores being proficient and fully 2/3 being accelerated or advanced, it still isn't enough to earn that "A", apparently  because the state realizes that with the cut scores being as low as they are, "Accelerated" or "Advanced" simply isn't good enough any more.

VALUE ADD

The next component is "value add" and this is the description from the waiver request:

Value-Added: While performance scores demonstrate a student’s level of proficiency, Value-Added
measures the effects of schools on their students’ growth. It is calculated only for schools with students in any Grades 4-8. Ohio, using the SAS® at EVAAS® model computes a Value-Added measure for each school and district in English language arts and mathematics and reports whether the expected growth has been met (a year’s growth in a year’s time), exceeded (more than a year’s growth in a year’s time) or not met (less than a year’s growth in a year’s time).

Interested readers can learn more about value-add here.

For purposes of this discussion, value-add was not changed in the new state report card. What was changed was the impact that value-add could have on the overall grade. Prior to the new state report card methodology, value-add was used to modify the overall grade. This was the algorithm used:

Value Add impact on final report card grade

With the new methodology, value-add is a full partner in the calculation. If you exceed Value-Add, you get an "A" for that component and the "A" will get averaged in to the final grade. This chart shows the new algorithm:

Thus, under the old system, it was possible to be "excellent" with a performance index of 90 or if you met 20 of the 26 standards as long as your value-add scores indicated "above expected" growth for two years running. Under the new system, that will no longer be possible.

ANNUAL MEASURABLE OBJECTIVES

The latest edition to the eduspeak vocabulary is "Annual Measurable Objectives" or AMO's.

The AMO metric is designed to replace the previous level of proficiency that was required by No Child Left Behind. NCLB required, by 2014, that 100% of kids achieve proficiency. Scholars can argue whether that was ever a realistic goal, but states eagerly took the money offered by the federal government back in 2001 and then back loaded the proficiency requirements. Fast forward 10 years, and as the mandatory levels of proficiency started to increase as we got closer to 2014, panic set it.  It is fair to say that the entire purpose of the waiver request is to relieve Ohio of the burden of a 100% proficiency requirement and the associated penalties when they are not met. The new goals, the "AMO's" are derived by taking the percentage of Ohio's kids that are currently deemed as not proficient in reading (18.1%), establishing a somewhat arbitrary goal of getting half of them to proficient within 6 years (e.g. 9% additional proficiency in 6 years) and using equal increments, so the bar is moved by 1.5% each year. This would then apply to all subgroups. A similar methodology is employed for the graduation percentage component of the annual measurable objectives. A somewhat complex calculation than goes into arriving at a final letter grade for the achievement gap component of the state report card.

The big difference between AYP and AMO's is this.  In order for AYP to negatively impact a final rating on the existing report card, you would have to miss AYP in the SAME subgroup for three (3) consecutive years. Of course, given the requirement to be 100% proficient, absent the waiver, that would happen to every district sooner or later, but not for several years. With the new methodology, AMO's are a full partner in the calculation and the failure to meet the annual objectives could impact your rating in the current year.

This new system will be in place, assuming the legislature goes along,  for the current year (2011-2012), so it is important to understand the differences between the old methodology and the new methodology, but it is also important to understand that the State Report Card will be changing fairly dramatically a few years from now to meet the demands of the assessment component of the common core.

The new methodology may therefore have a life span of a few years until the Common Core assessments kick in. Where this leaves Worthington is a bit unknown but as with most things, knowledge is power so understanding the methodology behind the new report card should give us a leg up on figuring out whether Worthington will be "A" rated or, if not, why not.

Grade inflation in the state of Ohio.

This post is about grade inflation in the state and the broader question of whether the rankings that districts receive on the state report card is truly reflective of the "quality" of the school district. In this context, grade inflation refers to the elevation, over time, of Ohio's School District Rankings and the number of districts receiving top marks from the state. The first thing we need to determine is whether grade inflation is real. Over the last year or so, an increasing number of articles, bloggers and reports seem to think so. Let's start with some statistics.

This was the breakdown in the 2004-2005 school year:

Where 18% of school districts were rated excellent and a combination of 67% were rated as excellent of effective.

Fast forward to 2011 where the chart looks like this:

Where 58% of school districts are now rated excellent or higher and an astounding 93% of school districts are considered effective.

Did Ohio really make all these academic gains or did the state manipulate the report card such that it now provides a distorted view of reality?

Let's look at a few other data points. The first comes to us from the Ohio Board of Regents where they track the success of Ohio graduates who go on to Ohio colleges and see how many of them require remediation - essentially, to retake high school courses when they get to college. If 93% of Ohio's School Districts were effective, you would anticipate this to be a relatively low number.

Unfortunately, it is not. NPR is reporting that 41% of freshman from Ohio High Schools have to take at least one remediation class once they get to college. The report goes on to say that an analysis of Ohio Board of Regents data shows that in nearly 80 high schools rated A or A+, at least half of students who enroll in Ohio public colleges must take remedial math and/or English.

That is an astounding statistic!  Your local high school is rated "Excellent" or even "Excellent with Distinction" and yet, half of its graduates have to essentially retake high school classes in order to be successful in college, yet, this applies to 20% of those districts with that excellent rating.

The Ohio Association of Gifted Children produced a report that was a less subtle about this problem. The report, entitled "Grading on a Curve, the illusion of excellence in Ohio Schools" made the case that grade inflation is taking place and that the state was essentially lying to its citizens about the quality of its schools. Charts such as this one:

compares the number of children scoring "advanced" on the Ohio Achievement Test vs. the number of children scoring advanced on the National Assessment of Education Progress (NAEP).

Obviously, Ohio is reporting a much higher percentage of children as "Advanced" then what you find on the national test. Of course, while Ohio isn't any better or worse than many other states, the state is using terminology that would seem to imply that it is.

One of the more egregious examples of how Ohio has created an illusion of excellence according to the OAGC report is through the use of "cut scores". A cut score is defined as the passing grade a student must achieve on one of the proficiency tests. The number that gets reported is that percentage of children that have achieved the passing grade, but up until this article, the grade that was considered passing never got any attention. The cut scores for the Ohio Achievement Tests in 2011 are documented in this chart:

So, for example, to receive a passing grade for 6th grade reading, a student must answer 17 out of 49 questions correctly for a total of 35%.  If 75% of the students accomplish that goal, the school or the district will be designated at having met that state standard. To receive a designation of "Advanced" in 5th grade math, a student must receive a score of 73%. The categories, proficient, accelerated and advanced all go into the calculation for the districts performance index.

The OAGC report and the Dispatch reporting  has numerous other examples of why the State Report Card is not a meaningful indicator of school district quality, so much so that the state has decided to take action. The next blog post will cover the new state report card and whether what was done will adequately address the problem.