“on grade level”

Elsie had a post about Goals, Objectives, Standards and the like that raises some interesting points. It made me realize that part of the problem is the woeful lack of understanding of statistics both in the general population and in professions who are using statistical information to provide advice (or even design regulations). So here is a little Statistics 101 conceptual refresher to help you deal with educational standards, like the definition of “on grade level”.

Developmental milestones, in health and in education, are based on statistical data on very large populations of children. They are averages of some sort. Those numbers are very meaningful for large populations, which can be expected to have about the same level of diversity as the even larger population from whom the figures are derived. So, for example (and using made up numbers), if you were to say that “on average” children learn to read by age 6, this would mean that in a very large population when you add up the ages at which children learn to read and divide by the total number of children, 6 pops out (the mean). That number could be the average even if NOT ONE CHILD actually learned to read at age 6.

You might actually use a different kind of average, the median. If the median age at which children learn to read is 6, that means that 50% of children learn to read before (and including) 6 and 50% learn to read after (and including) 6. If you strung all the kids in the population out in a line ordered by the age at which they learned to read, the half-way mark of the line would be a kid aged 6. Or between two kids aged 6. Or between a kid age 5 and a kid age 7 (unlikely to happen but it illustrates the point that this could be the right median even if no individual kids actually learn to read at 6).

With either measure you can see that those numbers can be correct, for a population, even if large numbers of children learn to read at ages ranging from 3 to 12. What counts as “normal” is not the same as “average” (by either definition). If the range of ages at which children learn to read and the percentage of children learning to read at a particular age remains relatively stable over a period of time, then learning to read at any age within that range is “normal”. Unfortunately, those statistics are much less interesting and harder to understand so they don’t get talked about in the media and by politicians very often.
So what use is the average? It might be nice to think that statistics lie, but that wouldn’t be true. A statistical average can be a useful measure when comparing large populations. So, for example, if you wanted to compare the quality of education between one state and another state, you could look at the average age at which children learn to read (or some other measure of attainment, ideally several). There are enough children of each age in each state to make that comparison meaningful. With that size of population, you could assume that the variation should be within the same range. So if it is not, you would want to figure out why. That might be because of the quality of education, but it could also be because of the quality of library provision, or the percentage of the population with English as a second language, or something else.

It probably also makes sense to use these kinds of statistical averages to compare school districts though some might be getting small enough that the differences from the average could be the result of random variation (in other words, not problematic). At the level of individual schools, the populations are probably not big enough for the statistical averages to be significant. Having one kid out on the far end of the range could skew the average significantly even though it is “normal” for a certain percentage of the population to be at that far end of the range. That doesn’t stop many jurisdictions, including the UK, from using school level standardized test results to rank the quality of schools. The fact that some authority does it, doesn’t make it meaningful.

Statistical averages tell you NOTHING meaningful about an individual. And if someone tells you that the quality of education you are providing is poor because one student is performing below average then they are misusing statistics to lie to you. If you know the normal range then that information can help you work out where an individual fits in relation to the population as a whole. Sometimes that information will be expressed as percentiles or deciles. So if your child is in the 60th percentile on some standardized test, it means that her attainment is higher than 60% of all the children who took the test. The median is the 50th percentile. Deciles cut the population into 10 groups. It is also helpful to know something about how the attainment varies between each decile because most of the population is likely to fall in a relatively narrow range, with fewer people at the outliers (the famous belled curve). So variation within the 5th decile might be minimal, but within the 1st and 10th decile might be considerable. The number of people within each decile is the same.

Do not get confused. If the results of some test say that your child is in the 60th percentile that DOES NOT mean that your child scored 60% on the test. It means that whatever your child scored was higher than what 59% of the test takers scored, and lower than what 39% of the test takers scored. Ideally, those percentiles will be calculated over a large enough population, like the whole state, or all individuals who took that particular test over the past 5 years or something.

Broader notions of grade levels are also based on some understanding of statistical average attainment for kids of a particular age. Ideally, those understandings should take into account the normal range of attainment within a particular age group. A good teacher should be able to work with reasonably diverse group in terms of ability and attainment. Unfortunately, the rise of standardized testing (and funding linked to performance in standardized tests) and ever more detailed “state standards” or “provincial curriculum” make it difficult for teachers to do this, as Elsie points out. It is important to remember that most school systems are graded by age, not by attainment. It is only a very small minority of children who will be held back; and a similarly small minority that will be accelerated. The combination of that with the narrowing of the acceptable range means that ever more kids need to be diagnosed with some “learning disability” in order get even some of their learning needs met. But that is another story.

For homeschooling parents, these notions are practically meaningless unless you are actually going to put your kids into school and need to know whether they are going to be ahead or behind the kids in their grade. What is important at the level of the individual, as opposed to the population, is whether a child is developing. Does your child know more now than she did 12 months ago? Has your child developed new skills or improved any skills? You only need to be worried if your child is stagnating or is going backwards. And that would be a trend over a period of time, not a few days. Development is notoriously uneven. You might also be worried if development is particularly slow. Slow development or lack of development might indicate an underlying problem that could be addressed. Development in some areas might happen more quickly or at different times than in another area. A certain amount of judgement, and perhaps advice from trusted others (including doctors, psychologists, etc) is required to work out if this is a “problem” or just what normal looks like for your kid.

The other thing this means is that, politically, it is in our interests to ensure that statistical measures are not being used inappropriately to judge us or to judge schooled children. If education policy misuses statistics to develop methods of working, regulations, test, funding formulas or whatever, that misuse will impact us. It may take a while. But if other kids are expected to meet certain milestones by a certain age, then sooner or later someone is going to ask that your kids be required to meet those milestones. And if it isn’t good for your kids, then you can bet there are at least some kids in the system who are also disadvantaged by it. Even if we don’t use them, public schools are provided on our behalf to provide an education for anyone who needs it. How much we do about it is up to us. But if we have opportunities to point out the misuse of statistics in formulating policy, we should take them. Even explaining the problem to relatives and friends increases the general understanding in the population. And challenging elected representatives who put forward these crazy proposals is probably a good idea.

This entry was posted in Homeschooling, Politics/Feminism. Bookmark the permalink.

5 Responses to “on grade level”

  1. willa says:

    I am guessing that some of the angst over OGL is the Lake Woebegone syndrome. Everyone wants their child to be “above average”, or if not, to have some certified diagnosis to explain why not. Moms even get angsty about children who are on the low end of the height/weight percentile or about age of toilet-training or first steps.

    I enjoyed the post very much. I think the political point is well taken, because educational policies affect society across the board, not just the “schooled” children.

  2. Angela, MotherCrone says:

    This is an excellent explanation, and one that I will be printing out to share with young homeschoolers who worry when Johnny isn’t reading by the end of kindergarten. Within a personalized curriculum, children can grow all over the board at their own pace. It is so freeing when you don’t let yourself get caught up in the grade level stuff!

  3. JoVE says:

    I think Willa is right. And I have seen the same thing about the infant height charts. I think that is also based in a misunderstanding of what they are, particularly the term “percentile”.

    The chapter in Angier’s “The Canon” on statistics is very good at explaining both the problems caused by widespread misunderstanding of statistics and the concepts themselves.

  4. wisteria says:

    Great post! I do think many get confused by statistics.

  5. shaun says:

    New homeschoolers often ask about testing requirements and usefulness. (Here, college-educated homeschoolers are required to do a yearly test of their choosing, though scores are not to be reported to anyone but the parent.)

    I try to point out that the first year isn’t going to be especially useful. The only useful info — and I don’t use it much! — is the difference between each year of testing. Is there movement, and (as a question to myself) why or why not?

    I admit I struggle with this — we have a child nearing 5 who is still not so good with the potty. But she is still in the “normal” range! Makes me nuts. . . Luckily she is the youngest, so by now I know that potty training this child is the least of my worries!