Categorie archieven: Uncategorized

Why Gene V. Glass quits the field of measurement

Gene V. Glass (1940) is a world famous researcher of education. His field for many decades was measurement. But in this post, he explains that he is resigning from his field.

Gene V. Glass
Gene V. Glass

“The degrading of public education has involved impugning its effectiveness, cutting its budget, and busting its unions. Educational measurement has been the perfect tool for accomplishing all three: cheap and scientific looking”

“Teachers and many parents understand that children’s development is far too complex to capture with an hour or two taking a standardized test. So resistance has been met with legislated mandates. The test company lobbyists convince politicians that grading teachers and schools is as easy as grading cuts of meat. A huge publishing company from the UK has spent $8 million in the past decade lobbying Congress. Politicians believe that testing must be the cornerstone of any education policy.

“The results of this cronyism between corporations and politicians have been chaotic. Parents see the stress placed on their children and report them sick on test day. Educators, under pressure they see as illegitimate, break the rules imposed on them by governments. Many teachers put their best judgment and best lessons aside and drill children on how to score high on multiple-choice tests. And too many of the best teachers exit the profession.

“When measurement became the instrument of accountability, testing companies prospered and schools suffered. I have watched this happen for several years now. I have slowly withdrawn my intellectual commitment to the field of measurement. Recently I asked my dean to switch my affiliation from the measurement program to the policy program. I am no longer comfortable being associated with the discipline of educational measurement.”

Time to move beyond test-focused policies

An anti-testing backlash has emerged among parents in the US demanding changes to the No Child Left Behind Act (NCLB). 


The NCLB  Act

NCLB supports standards-based education reform based on the premise that setting high standards and establishing measurable goals can improve individual outcomes in education. The Act requires states to develop assessments in basic skills. To receive federal school funding, states must give these assessments to all students at select grade levels. The Act does not assert a national achievement standard. Each individual state develops its own standards. NCLB expanded the federal role in public education through annual testing, annual academic progress, report cards, teacher qualifications, and funding changes.

Yet, as a new National Education Policy Center Policy Memo published today points out, the mistakes in NCLB are still being repeated, and lawmakers’ discussions in Washington, D.C., surrounding reauthorization of the law are failing to adjust course.

Tests are ineffective and counterproductive when used to drive educational reform

NCLB was “an ineffective solution to some very real problems,” according to the new NEPC Policy Memo. The memo discusses the broad research consensus that standardized tests are ineffective and even counterproductive when used to drive educational reform. “The problem is not how to do testing correctly. In fact, today’s standardized assessments are probably the best they’ve ever been. The problem is a system that favors a largely automated accounting of a narrow slice of students’ capacity and then attaches huge consequences to that limited information. Testing used as a diagnostic or summary instrument for children’s learning can be a helpful tool. It is harmful, however, to use students’ test scores as a lever to drive educational improvement. This use of testing is ill-advised because it has demonstrably failed to achieve its intended goal, and it has potent negative, unintended consequences.

Thirteen NCLB years of intense focus on test-score improvement has yielded few if any benefits. Yet negative, unintended consequences have continued to mount—in the form of narrowed and less engaging curriculum, constrained instruction, and deprofessionalized teachers and teaching, the memo points out.

”We see clear trends of abandoning our past pursuit of learning that fully encompasses arts, music, social studies, and science; and we see marginalization of values and skills that help students develop the ability to cooperate, problem solve, reason, make sound judgments, and function effectively as democratic citizens.The ultimate question isn’t whether test scores are good measures of learning, whether growth modeling captures what we want it to, or even whether test scores are increasing. It is whether the overall impact of the reform approach can improve or is improving education.”

The memo points out that test scores “can be increased in lots of different ways, some of which focus on real learning but many of which do not. An incremental increase in reading or math scores means almost nothing, particularly if children’s engagement is decreased; if test-prep comes at a substantial cost to science, civics and the arts; and if the focus of schooling as a whole shifts from learning to testing.”

The NEPC Policy Memo, Reauthorization of the Elementary and Secondary Education Act: Time to Move Beyond Test-Focused Policies, can be found on the NEPC website at

Better data would lead to better decision-making around school choice?

The American Enterprise Institute (AEI) recently published the report, Better Data, Better Decisions: Informing School Choosers to Improve Education Markets. The AEI argues that better data would lead to better decision-making around school choice for parents, presumably strengthening good schools and allowing families to steer clear of low-quality schools. This report was reviewed by the National Education Policy Center

The AEI report provides a thoughtful reflection on, and assessment of, school choice decision-making. It reviews a number of different efforts to engage stakeholders in the school choice process. Ultimately, while the report does offer concrete suggestions aimed at improving the quality of school choice decision-making, it does not successfully argue that increasing choice will increase school quality overall. The report does recognize that many school-choice decisions are made based on informal networks and criteria that researchers have had difficulty identifying and evaluating.

This 16-page report (Better-Data-Better-Decisions)  asserts that choice is seen as a mechanism, and at times a panacea, for better educational quality. Given the intense interest in choice as a reform strategy, the report promises to provide an “empirical look at how governments and other third-party organizations can help inform families about their school choice options” (p. i). The report considers how to better inform choosers. Recognizing that no choice is ever perfectly informed, the report provides background on choice theory, focusing on the notion of “bounded rationality.” What this means is “decision making happens in contexts in which people lack relevant information about their options, have finite time to commit to collecting that information and are limited in their abilities to process and use what information they have” (p.2). The author argues that school choice is not an entirely “rational” decision but is shaped by 1) the context within which the choice is made, 2) the background of the chooser, and 3) the available information.

The report provides an overview of what families want from schools, where they get information on schools, and how they use information to make decisions. This section of the report relies on research conducted by the author. He notes that most information on what families want from their children’s schools is based on survey data which, while useful in some respects, does not take into account desirability bias: the tendency of respondents to respond in ways they think they are “supposed to.” The report suggests that other factors (such as accessibility and information from social networks) are actually more influential in real-life decision-making. It further argues that preferences and desires are malleable: they can change over time and in light of available information.

The report asserts that research on school choice has described two main sources of information: social and formal. Parents talk to other parents and make decisions based on these conversations. Some parents also consider other sources of information, including school districts, governmental entities, and non-governmental third parties. Two critical points the report makes about families’ ability to gather information are: 1) the quality of information available through social networks is heavily influenced by social class, with more well-to-do families typically having access to better information, and 2) the way information is presented (timing, format, amount, source) influences the capacity of the student or parent to use the information effectively. The report notes that finding ways to cut through “data smog” to determine what information is reliable, good, and relevant is a primary challenge. Lastly, the report addresses the challenge posed by the fact that, especially for middle and high school students, there are multiple decision makers (parents and students) who come to the process with different informational needs and desires. The report addresses the challenge of helping parents and students make good decisions that prioritize academic quality.

The report concludes with a series of recommendations presented in the language of bounded rationality; that is, the author advocates solutions that take into account the imperfect human reality of decision-making and that improve the odds that parents and students will make “good” decisions by providing them with good information in a timely fashion in an accessible format. The report encourages families to actually make a choice— to consider the myriad options available to them rather than “choosing” those most readily accessible (convenient) to them or passively accepting the schools selected by others in their social network. Those interested in improving student and parental decision-making are encouraged to be accurate in their information, to provide it in an accessible format, and to accommodate the imperfect reality of real people making hard choices that balance many facWindow_en_www_aei_org_wp-content_uploads_2014_11_Better-Data-Better-Decisions-4_pdftors beyond a school’s academic quality.

AEI’s Better Data report is grounded in both the extant literature on school choice and the author’s own research. Beginning with a useful introduction to the concept of “bounded rationality,” the report goes on to review recent research on choice generally and school choice specifically. The report relies on the literature from psychology, economics, and education to provide the reader with an understanding of work on choice and decision- making generally as well as on school choice in particular.

The author presents findings from two studies he conducted: an Online Experiments Study and a Field Experiments Study. The methods and findings for both studies are presented in sepfieldarate text boxes in the report, while conclusions drawn from the research are embedded in the report.

The Online Experiments Study used an online survey of 1,000 adults to understand how individuals use numerical ratings of components of school quality and parent comments.