EduGuru

America Competes Act - 12 Elements for Data Systems - Statewide Student Identifiers

The Feds are going to require all states taking stimulus money for P-12 education to have 12 “elements” in their statewide education information systems.  The EduGuru is going to examine each of the 12.  In case you’ve been in a cave for the past few years, the first 10 elements are the components from the Data Quality Campaign recast in the America Competes Act.  Nothing bad there (see The Process for Ensuring Data Quality, ESP’s Optimal Resource Book for a review).

The Eduguru rates each element on two scales from 0 (lowest) to 10 (highest).

• How important is it for the Feds to track this? (So what?)
• What’s the likelihood the states are going to pull this one off?

Then the Eduguru makes a summary declaration.

• What’s the bottom line on this one?

Number 1:  A Unique Statewide Student Identifier

• How important is it for the Feds to track this? 1
• What’s the likelihood the states are going to pull this one off? 10
• What’s the bottom line on this one? Save everyone’s time reporting this one.

The unique student identifier must be assigned statewide and “not permit the student to be individually identified by users of the system.”  What?  Isn’t that the purpose of a unique identifier?  In this case the point is to allow a student’s records to be matched up across years and across files without a viewer of reports being able to figure out that student’s personal identity.  That can be done these days.

In fact count this one as done in all states.  Maybe not done completely, but there’s no argument about this one being reality.  The challenge will be whether or not the safeguards will protect every student’s identity.  The betting line?  There will be problems through carelessness.  Get ready for them.

The EduGuru speaks:  Glad this is number one, so we can begin with a victory.  Each one gets harder as we go. Next post will address Student-Level Enrollment, Demographic, and Program Participation Information.

Diagnosis for Teachers.

With the changes in the Executive Branch, NCLB will change, but how?  The Eduguru is continuing a series of blogs on the problems opponents have had with NCLB and what they might or should do about them now.

The seventh is focus:  Diagnosis for teachers.

The annual statewide assessment for accountability is not for teachers.  That’s correct, depending upon this high-level, annual assessment to provide teachers useful diagnostic data is silly.  The focus of these assessments is accountability.  Teachers need a separate set of assessments for diagnosis.  See Why Eva Baker Doesn’t Seem to Understand Accountability, ESP Solutions Group Optimal Reference Guide.

NCLB naively requires timely reporting to teachers for diagnosis.  That’s fine, but ask a teacher how useful the state assessment results are to them.  Interesting, yes.  Important, yes.  Meeting their needs for diagnosis, no.  Psychometrically, what makes for a great accountability test does not make for a decent diagnostic assessment.

Congress should authorize separate diagnostic assessments if they want teachers to benefit from the statewide assessment programs.

The EduGuru speaks:  Seen the satellite photos of your house and neighborhood?  Great for judging where you are relative to community services and stores.  Not very useful for determining what renovations need to be made.

Timely Reports for Parents.

With the changes in the Executive Branch, NCLB will change, but how?  The Eduguru is continuing a series of blogs on the problems opponents have had with NCLB and what they might or should do about them now.

The sixth is practical:  Timely reports for parents.

The aspect of NCLB that may be the farthest from being implemented well is the timely reporting of AYP results for parents to make school choice decisions.  The culprit is the long cycle time for scoring assessments.

Pushing the testing dates earlier in the school year diminishes the validity of the results for parents.

This requirement is one that should not be changed.  The states and assessment companies need to rethink their processes to reduce the cycle time for scoring and reporting.  Granted, all states pile up with their testing and scoring needs during the same time period.  How can Texas get millions of tests scored and reported back to schools within a few weeks and other states require much longer?  Negotiation skills and money.

The states with the best cycle time are those that manage their processes on the front end the best.  Pre-coding accurately and shortly before testing.  Using the assessment contract as a vehicle for scoring, not for collecting demographic and other student data that should already be in the state’s data store.  Ensuring schools provide compliant answer documents that will not require trouble shooting by the scoring service.

The Eduguru speaks:  There’s plenty of money in the system for timely scoring and reporting.  This is a case where processes need to be improved to reduce cycle time.

Grade Levels Assessed.

With the changes in the Executive Branch, NCLB will change, but how?  The Eduguru is continuing a series of blogs on the problems opponents have had with NCLB and what they might or should do about them now.

The fifth is expensive:  Grade Levels Assessed.

What if we add more grade levels to the accountability system?  Some grade levels (e.g., PK-2, 11-12) are not formally assessed for inclusion in AYP.  Some sort of measurement is required, especially for schools with only these unassessed grade levels.  Adding appropriate assessments statewide would be expensive.  Continuing to leave them out makes for a spotty accountability system.  This is especially true where student performance is being linked to teachers for pay-for-performance or annual evaluation.

Alternative measures are available, but typically are even more expensive, unreliable, and difficult to quantify (e.g., teacher ratings of students, individual reading inventories, performance measures graded by the teacher).  Eva Baker, past AERA President, has recommended a new generation of performance measures that will take years to develop, validate, and standardize.  States could merely be tasked with the obligation to figure this all out.

Growth measure advocates require multiple years of preferably adjacent grade level testing for their calculations.  If growth can’t be measured until the end of grade 4 or after grade 10, then the two spans that get the most attention in the research are missing.

The Eduguru speaks:  The NAEP-sayers must admit their omnipotent assessment falls short here also.  Will on-line assessments arrive in time to solve this dilemma?

Proficiency Across States.

With the changes in the Executive Branch, NCLB will change, but how?  The Eduguru is continuing a series of blogs on the problems opponents have had with NCLB and what they might or should do about them now.

The fourth is politimetrics:  Proficiency across states.

The contrasts in definition of proficiency across the states has been one of the most fascinating aspects of NCLB.  States rights advocates applaud allowing each state to adopt its own standard.  Of course, NAEP is the only practical and available basis for judging relative difficulty of proficiency cut points.

Proficiency will be defined more normatively—across states—possibly demanding an alignment with NAEP.  This is not as difficult to pass as you might expect considering that many of the states that have the lowest standards for proficiency and the lowest NAEP scores voted Republican.

Debates have raged for decades as districts and states adopted their own tests called minimum skills, basic skills, essential skills, grade-level skills, exit-level skills, and many other terms to designate that what’s measured is somewhere between the lowest level acceptable or a high standard for excellence.  NCLB created proficient as the goal, expecting it to be a high standard but not requiring it to be as high as NAEP.

Politimetrics, the combining of psychometrics and politics to make decisions, will determine the definition of proficiency and how it’s aligned across states.  Look for this to be one of the liveliest debates.  One current strategy of some, following NCLB but having local or state accountability ratings as well, will get more difficult to defend.  As NCLB softens up in other areas, this proficiency issue may get more controversial.  No state can publicly adopt a local proficiency standard that is below the Federal standard.  Having one that’s different in how it’s calculated or what it includes is fine, but expecting less of our proficient children is not going to work.

The Eduguru speaks:  What if NAEP became the standard, but NAEP’s Basic Level became the criterion?  Then states can be encouraged to adopt standards higher than basic—if they want.

100% in 2014.

With the changes in the Executive Branch, NCLB will change, but how?  The Eduguru is continuing a series of blogs on the problems opponents have had with NCLB and what they might or should do about them now.

The third is an easy one:  100% in 2014.

No one believed this goal would be reached, but it did set a long-term target that influenced annual objectives.  Besides, how do you argue against wanting every student to succeed?  Which ones are we willing to leave behind?

Look for Congress, the states, or the Education Secretary to make these changes.

1. Proficiency will be defined more normatively—across states—possibly demanding an alignment with NAEP.  This is not as difficult to pass as you might expect considering that many of the states that have the lowest standards for proficiency and the lowest NAEP scores voted Republican.

2. Growth will become the new controlling factor.  The indication that standards are falling would be that growth is no longer defined as being on pace for proficiency by the end of n years.  If any measurable, reliable growth becomes acceptable, then the barn door will be opened.

3. An IEP for special education or English language learners may trump test scores as the measure of growth.

4. Full academic year will be replaced by a longer term to reduce the number of low performers in the mix and to allow for them to recover using growth.  Look for a two-year term to be used to determine which students are included in AYP.

5. The Education Secretary may adopt a national standard for small subgroups to level that issue.  Look for the subgroup size to be 30 or above, or to be based upon confidence intervals that might require even larger groups.  (This is for AYP, not for reporting descriptive statistics for annual public report cards.)

6. Will subgroups survive as a disaggregation criterion?  Look for alternatives such as majority/minority.  Keep in mind that the new President fits into multiple race categories.  Maybe race goes away and is replaced by income.  Now that would be consistent with campaign talk.

7. Reporting 100%?  Congress may want to clean up the FERPA conundrum of not allowing reporting of 100% categories, unless new guidance is deemed sufficient.

8. Requiring 100%?  Very difficult to keep.  Very difficult to abandon.  100% can be kept as a distant target (some states already do this)—one that continues to move out into the distance as we approach it.  Like a mirage that disappears when you approach and reappears farther down the road.  Look for a 5 to 10 year horizon for 100% instead of a specific year—if 100% survives at all.

The Eduguru speaks:  I’m 100% certain there will be multiple changes, the combined impact of which will not be known for years.  Unless…Congress could authorize some impact studies for rule changes—after all, the states have longitudinal data now with which to test the proposed changes.

Including the Disabled in Accountability.

With the changes in the Executive Branch, NCLB will change, but how?  The Eduguru is continuing a series of blogs on the problems opponents have had with NCLB and what they might or should do about them now. 

The second is a touchy one:  Inclusion of the Disabled.

NCLB required that the disabled students be included in the accountability process.  Much delight was expressed by the parents of those students who before had been excluded from accountability processes and now have the extreme attention of educators who see many of their schools being designated low performing only because the special education student subgroup did not meet adequate yearly progress (AYP).

Relaxed guidance allowed more students to take alternative assessments.  All quota systems with a national percent cap are difficult to implement at the local level.  Local schools don’t all have a perfectly random sample of special education students matching the national numbers. 

The NCLB requirement that all students be tested on grade level impacts the special education students directly.  A grade level test may be frustrating with a preponderance of items that are too difficult for students performing below grade level. 

The challenge is how to include all students in the accountability system, but do it fairly. 

The EduGuru speaks:  The parents of special education students will rule this issue with a sympathetic new administration.  Look for special education to no longer be a subgroup (assuming the whole subgroup notion survives) with the same rules as others.  Look for Congress to give the Secretary and/or states more leeway, but ensure special education students are a part of the formula for accountability.

Change Impacts NCLB.

The Nation’s Children Love Barak, that’s not the NCLB we are going to talk about changing.

With the changes in the Executive Branch, NCLB will change, but how?  The Eduguru is starting a series of blogs on the problems opponents have had with NCLB and what they might or should do about them now. 

The first is the easy one:  Underfunded.

Congress will need to find that extra funding someplace among the bailouts.  The dollars don’t appear to be available from a windfall profits tax on big oil with gasoline below $2 a gallon again.  The reality is that Congress typically funds their own projects at a level lower than is authorized in the enabling legislation, so NCLB is not that unusual. 

Understanding what is meant by NCLB being underfunded is tough.  Just saying more money for schools in need of improvement is too vague.  Title I has been funding them for decades—and the amount of unspent Title I dollars each year raises the question of the ability of those schools to wisely use even more money.  With the complaints about the What Works Clearinghouse, where will the new administration find worthy programs on which to spend millions of dollars? 

The dollars appropriated for parents to find alternative tutoring and services for their children have been largely unspent—partly because schools discourage parents from seeking external help, and partly because the districts and states haven’t identified and publicized the alternatives as well as they could have. 

Let’s assume Congress provides more dollars for the Elementary and Secondary Education Act, aka Improving America’s Schools Act, aka the No Child Left Behind Act.  How will they be allocated to states, and what restrictions will accompany them?

The EduGuru speaks:  Education wasn’t much of an issue during the election.  Looks like GM/Ford/Chrysler, mortgage lenders, and another round of stimulus checks will get the first money.  Low-performing schools don’t cost us money—directly.  They don’t depress the economy—directly.  They don’t reduce our savings—directly.  So, Congress will likely not get to them directly—as they say in the South.

Everyone who passed the test, get on the bus. Sorry, Charlie, only the best tunas get to go to Sea World.

The Texas Education Commissioner had to remind everyone that publicly rewarding students who pass the state test violates FERPA.  (Associated Press, October 14, 2008)  Ummm…what about those National Merit finalists, the magna cum laude graduates, the honor roll students?  All those students not wearing the gold tassel or the gold sash at graduation failed—to a degree.  They don’t get to go on the bus trip to the elite universities.

Should the high school yearbook include each student’s height and weight along with clubs and honors?  

How did we get so differentiated in our proclamations of success and failure between academics and sports?  Is there something personal or embarrassing about being slow in the classroom more so than slow on the field?  We publish athletes’ height and weight, their speed, their won-lost records.  By the way, the local school board must still designate sports vital statistics as public to follow FERPA.

The EduGuru speaks: School is about the curriculum, so the smart kids rule over the others. Even FERPA doesn't trump the naming of the valedictorian. Sports are extracurricular, that is "less" curricular, not "even more" curricular.

We’re taking our school voucher to Korea!

U.S. students rank below 14 other nations? True. However, do you really believe that? Even in the presidential debate, the moderator quoted the miserable ranking of U.S. students among all nations of the world. We spend more and achieve less.

Is the comparison fair?  Our public schools include everyone—almost.  We test them all—thanks, NCLB.  We even graduate 80% of them—including the special education students with modifications.  Our English language learners have to be tested in English—do other countries test only in their official language?  Our schools teach their own state’s standards—not some international panel’s opinion of what’s important globally.  Our school day is more reasonable—shorter.  We rely inordinately upon homework—actually less than other countries.  We educate the illegal immigrants—shouldn’t their scores be assigned to their home countries?  We take the summer off so kids can help their parents back on the farm—or travel to those other countries.  We blame ourselves for global warming and teach green—well, they blame us too.  Our course catalogues list more elective courses than solids—does anyone still use that term?  The Pope should take responsibility for the Catholic schools.  Charter schools aren’t doing well enough.  College engineering schools should give preference to Americans.  Did you see the last National Spelling Bee?—our students are from those other countries.

If we had universal vouchers, would parents send their kids to other countries? We can make too many excuses, but if you believe in face validity, those rankings just don’t add up to us.  Are our U.S. members on that panel really representing us well? 

The EduGuru speaks:  How in the world can they determine what someone in Beijing, Beruit, and Baytown should know and measure it fairly?  Is too much faith being placed in those tests and the standardization of them?  The U.S. doesn’t have to be first, but are those other countries really that good?