Speechwriter: Warren Goldie & Chief of Educational Accountability
Speaker: Chief of Educational Accountability, Baltimore City Public Schools
Conference: National Center for Educational Statistics
[SLIDE: TIPPING THE SCALES]
Thank you for inviting me. Thank you to the National Center for Education Statistics for all your hard work in bringing the conference together. The amount of information available in these few days is just phenomenal.
[Thanks to previous speaker and anyone else on stage]The Elementary and Secondary Education Act of 2001 — also known as the No Child Left Behind Act — places unprecedented importance on data in determining education outcomes. This constitutes a significant shift of perspective from previous thinking.
The question is, How will such a shift affect school districts? Will the new, more stringent reporting requirements resulting from No Child Left Behind change state and local data systems? How will education policies and practices be affected?
These are some of the issues we’ll look at today—from the perspective of a large urban school district’s student assessment, evaluation, and accountability office.
I’ve titled this presentation “Tipping the Scales” – as we add data on one side, student achievement rises on the other. How does having more data increase student achievement? That’s the main question I want to approach today.
[SLIDE: KEY QUESTIONS ADDRESSED]As for those of you who are data collectors, I will also address the following questions:
Under the new legislation, what will be required from our data reporters?
How can we provide the best information possible to help in state education policy and decision-making, as well as in classroom instruction?
In order to address these questions, I’m going to take a three-pronged approach. I’ll start with an overview of the XX School System and its experiences. Where we’ve been, where we are today.
Unlike many other urban districts across the country, we have been striving for reform for some years now. I’ll speak a little about that and how it’s been going. Next, I’ll share the role of my department, which is responsible for student assessment, program evaluation, institutional research, and strategic planning – and what it’s like to do all this in an urban district that is undertaking significant reform initiatives.
And finally, the major focus of my presentation will be looking at how, with the passage of No Child Left Behind, student assessment, program evaluation, research and data needs, and strategic planning will have to change.[SLIDE: YES/NO TEST TAKING]
Before I start, however, I would like to administer a brief test. Yes, you heard me right – I said test. So, grab a piece of paper … No, you don’t need a number 2 pencil!
You didn’t think this was going to happen, did you?
Okay, ready? Good. Now, please number the paper from 1 to 5.
Now unlike the S.A.T., you will not be penalized for wrong answers. And I would also ask that you take the N–C–E–S Data Conference Pledge not to cheat off your neighbor!
Please respond to each question with a “Y” for yes or an “N” for no.
Okay, let’s begin. You may figure out what I’m getting at here, but please, follow me to the end anyway. There’s a method to my madness.
Number 1. Yes or No: We can gauge the academic performance of public school students through examining achievement patterns derived from testing programs. Please jot down your answer.
Number 2. Reading Literacy and Mathematics are among the most important skills taught in public schools. Yes or No?
Number 3. We should include as many students as possible in our testing programs. In other words, we should not intentionally exclude students whose scores may bring down the overall score for a school or district.
Here we would have to accept that we can’t always test 100% of the students. For example, we’d have to exclude Jeannie who is undergoing chemotherapy treatments for cancer, or Sue who we are trying to teach, at 16 years old, how to self-feed. Yes or No?
Number 4. It is important, in examining test performance, that over time, we see an increase in student performance. Yes or No?
Number 5. A teacher who is qualified as demonstrated by good teaching skills, good knowledge of content, and an understanding of pedagogy — that is, how to teach — along with an enthusiasm to work with young people … is generally better than one who does not have training or experience in these areas? Yes or No?
That’s it. Thank you. Congratulations! You survived my test. Now let’s examine the answers. If you had at least three Yes’s — that’s three or more — please raise your hand.
Great. Thanks.
If you said Yes to all five questions, please keep your hands up. Otherwise, put them down.
Thank you. Now look at that. As I anticipated, many of us still have our hands up. That’s because, while we differ in many ways, come from different parts of the country, have different roles, originate from various educational training levels and age groups … we agree on many of the same underlying principles.
Great. Thank you for your help. Please put your hands down.
One of the foundation concepts that we agree on is the need for accountability in our schools and programs. Another one is the importance of using student achievement data in decision-making. We see the need for the highest-quality teachers in our classrooms, that resources are based on research, and are focused and targeted on the neediest students.
Briefly stated, the four basic education reform principles of No Child Left Behind are:
Stronger accountability
Increased local control
More options for parents
Better teaching.
I would respectfully submit that the short test that we data folks just participated in provided us with, well … data! An overwhelming number of us agreed to the majority of five statements that capture the underlying essence of the No Child Left Behind bill …
… even though we may not agree with all aspects of it …
… and even though we may not all understand what is meant by certain aspects of it ….
.. and even though we may not yet know all of the repercussions of it.
We do, as a group, tend to feel that schools and students should be held accountable for their performance. And that one way to measure student performance is to examine test data. And further, that having highly qualified teachers in our classrooms is vital to our success.
[SLIDE: XX Org]Having set that stage, I would now like to briefly highlight for you the reforms taking place in the XX Public School System.
XX city is a district of 173 schools serving 92,000 children.
Most of them come from low income, minority families. I won’t beat around the bush. We are a poor system. In the past, we suffered from chronic low achievement. We’ve experienced all the challenges you might expect. Many of our schools had fallen into a status in our state known as reconstitution.
[SLIDE: ENROLLMENT]We have had difficulty attracting teaching talent. Over the past few years, about 40% of our teaching force were not certified. Our student enrollment has diminished. Our management systems were problematic. Our physical buildings need work.
Our schools were faltering badly — and people knew it. Fortunately, awareness of this problem finally reached critical mass in the greater community. The situation could no longer be left alone.
Five years ago, the XX state government decided to do something about it. In 1997, the state legislature created a city-state partnership, drawing the best leadership from the city and state to share power and find a way to reform the school system.
[SLIDE: CITY STATE PARTNERSHIP, MASTER PLAN]Control of the schools was transferred from the mayor’s office to the new leadership, made up of education professionals from both the city and state. 254 million dollars were budgeted for a 5-year reform effort. It produced a strategic plan known as Master Plan I.
[SLIDE: TWO GOALS OF MASTER PLAN]Master Plan I was a blueprint for change. It had two overarching goals: improve student achievement and overhaul current management practices. The master plan strategy was informed at every level by the underlying philosophy of using data to improve student achievement. Master Plan I was implemented in the 1998-99 school year.
[SLIDE: SIX OBJECTIVES OF MASTER PLAN]In addition to its two goals, Master Plan I defined 6 objectives.
But critically, Master Plan I kicked off the era of data-driven decision-making in XX City. Prior to this, we would provide data to the state for one reason and one reason only: to meet the state’s data reporting requirements. We were not involved any further than that. Now, that’s changed. Now we affect policy and practice.
As in No Child Left Behind, plenty of accountability was built into Master Plan I. Accountability was crucial. We were under the microscope all the time — with the legislators, the people in the school system, parents, the public.
Everyone wondered: Could we actually turn the city’s schools around? There was also high visibility in the press — articles written about “XX’s ambitious school reform efforts” … and scrutiny from all quarters, on both the pro and con sides.
[SLIDE: TERRANOVA RESULTS]Well, the reforms must have worked. The XX School Performance Assessment program, which we call XX, has shown consistent improvement.
Our TerraNova results— the tests taken by 1st to 8th graders — have shown improvements at every grade level. Under Master Plan I, many of the most dramatic improvements came at the elementary school level.
[SLIDE: WESTAT EVALUATION]An evaluation by Westat confirmed all the positive strides we had made, and continue to make. The Westat evaluation addressed four questions:
Has student achievement improved? Westat says yes.
Have we improved system management, and are management tools being used effectively? Yes.
Are improvements the result of the partnership? Yes.
Were the funds provided sufficient for the reform. To this Westat said no; more funds were needed.
In 2002, after five years, we came to the end of Master Plan I. Our state legislators then approved another 5-year plan. Master Plan II was essentially an extension of a plan that was already working, with some important changes.
As before, the emphasis was on continuing to use data to elevate student achievement and improve management systems. The first master plan has its best results at the elementary school level.
Master Plan II targeted secondary schools more. But it continues to work at the elementary level as well.
Now I want to say a few words about my department. DREA. That’s D-R-E-A. The Department of Research, Evaluation, and Accountability.
[Slides about DREA]Now, whether we are speaking about our Master Plan or about No Child Left Behind, one point is clear: Whoever is working with data in the school districts has become more important than ever.
The new legislation places an unprecedented importance on data: data that measures school and student performance. Personally, I try never to lose track of the highest purpose of this data, which is that it must function in the service of our students, to create better educational outcomes for them.
I called this talk “Tipping the Scales” to show that data and student performance are both not only critical to success, but closely related to each other. The more we use data effectively, the more we can increase student achievement and help students.
How will No Child Left Behind affect the XX public school system and urban systems like it? One way is through its accountability requirements.
Under N.C.L.B., all states must implement accountability systems, setting standards in each academic content area. States must gather objective data through tests that are aligned with those standards. This test data will be used to identify strengths and areas in need of improvement in the system and thus – hopefully – bring about positive changes in schools.
An integral part of my department’s accountability effort is our Student Assessment Team. How will its efforts change under No Child Left Behind? There are many ways.
We will:
Implement new state testing program
Realign local testing programs
Provide lots professional development and support on new programs
Provide data disaggregation
Due to No Child Left Behind, our student assessment teams will have to disaggregate data on student performance data more than we have in the past. Data mining will be more intense than ever. The disaggregation required by the legislation goes deeper than ever before—as deep as six levels down.
State and district reports will now show parents highly detailed information on their sons and daughters. These reports will shed light on achievement gaps between students of various groups, such as the economically disadvantaged, racial and ethnic minority groups, students with disabilities, those with limited English proficiency, and so on.
Looking at these subpopulations and their achievement gaps will help close the larger gap and in the spirit of the legislation, ensure that “no child is left behind.”
[HOLD UP 2,000 PAGE TERRANOVA DOCUMENT]This 2,000 page document is XX Public Schools’ disaggregated TerraNova report.
So, where are we today? Where is my division?
We are a producer of data in an NCLB climate that seeks data. That makes us very popular. We have a higher profile than ever before.
We are also unpopular — if we can’t produce data fast enough, or don’t have all the answers, or can’t meet Adequate Yearly Progress standards, or don’t have the numbers that show what we need them to show … well, you can see what I mean …
Many of us here agree with the underlying principles of the No Child Left Behind legislation, but we may not agree with all of its aspects.
AYP — Adequate yearly progress — is the way that the legislation measures success or failure. We know that these standards have yet to be defined, and this is an area of contention at this time. But whatever they eventually are, we know there will be consequences for schools that don’t perform up to par.
Urban schools like ours are often targets for legislation like No Child Left Behind. The reason is easy to figure out — our schools don’t perform as well as others — but there is an explanation:
We work with the most challenging student population. My personal opinion is that highly punitive policies directed at troubled schools through NCLB may be too harsh. It would be better to study the data, see where the problems lie, and then work to correct them.
NCLB calls for much change. Change can be hard. School systems are scurrying to understand its requirements and find ways to meet them. But it should be noted that there is danger if change comes too fast.
[SLIDE: “ANXIETY LEVEL” GRAPH]You are all familiar with the bell curve. This is what we call the anxiety level graph. Note the axes. On the left, we have the level of anxiety. On the bottom, the number of changes.
Implementing the changes required by NCLB will cause anxiety in school systems. Any big change will do that. You can see that a moderate stress level in implementing NCLB is good, but a low or high level is bad. The lesson here is that too many requirements by the government means diminishing returns.
[SLIDE TO END: BALANCE SCALES]I began by asking two questions. What would be required by our data reporters? And how can we provide the best information possible to help in state education policy and decision-making, as well as in classroom instruction?
[Ending TBD]