Monday, 1 June 2015

Evaluating Year 12 results (2015 Term 2 Week 7)

One of the key influencers for families in their choice of school is the final academic result. We all want our children to do well, in order to maximise their opportunities, primarily with reference to university entrance. It is a legitimate goal for families, and for schools, to want students to achieve strong results. For many, the HSC results constitute a verdict on the quality of the education at a school.

At this point, I find myself in an awkward stance. As Principal I obviously have a vested interest in putting a positive spin on Inaburra's results, and any comments I make regarding evaluation may be read with suspicion! Yet as an educator (and a parent), I want to be sure the results can be evaluated effectively and fairly. Apart from anything else, rigorous evaluation is a necessary step in any improvement agenda!

In 2014 Inaburra School students attained fewer Band 6 results in their HSC than they had done in the previous few years. This was disappointing to us, but not surprising. We know our students well and we understood the level of their performances, throughout their time at school. We rejoiced with them at many of the results that they achieved; we also shared the disappointment that some of them experienced.  

However, I do not accept the proposition that I have heard that our students' academic performance fell. Nor is it valid to suggest that the academic standards of the school are sliding. Let me explain.

An easy way to evaluate?
In the newspapers each year, and on a number of websites, it is possible to find tables that purport to rank and compare schools. The measure that they use is to divide the total number of HSC exams sat by the total number of Band 6 results (marks over 90). This result is used to make a simple ranking table. This method appears to have the great benefits of being simple, clear and objective. Moreover, it seems relatively simple to compare a school's ranking one year with its ranking in another year, so it is possible to gauge a school's trajectory - up or down!

Whilst an apparently simple, clear and objective ranking table may be easy to understand, it is a very superficial reading of limited data and, as such, it can be a poor guide to the academic performance of a school.

The popular method of ranking schools used by the media and websites (described above) is the only way they are legally allowed to make comparisons. Section 18A of the Education Act 1990 prohibits the public comparison of HSC results with reference to anything other than marks over 90; that is, Band 6 results. To compare on any other data publically is prohibited. This is a political decision, not an educational or statistical decision.

What other options are there?
Using only the number of Band 6s as the basis for comparison is a very narrow measure. It would be analogous to evaluating the weather only with reference to the number of clear days. It is valid to count the number of clear days, but evaluation should also include the rainfall, temperature range, wind, patterns, trends, averages, atypical weather events and whatever other measures are available and helpful.

There are many other ways to evaluate HSC results. For example: 
  • the average mark received by students at a school in each subject, compared to the State average
  • the profile of Band results compared to the State (what percentage got Band 6 in a specific subject, what percentage got Band 5, what percentage got Band 4 etc); 
  • the distribution of Band 6 results across a cohort; that is, are the Band 6 results concentrated amongst a small group of students, or are they disseminated more widely across the year group.  
  • the distribution of results by subjects also yields valuable data by indicating if there are some areas that are stronger or weaker than others.  
  • the value-add, whereby we identify each student’s performance at a prior point, project their expected results and then measure to see if those predictions are reached, exceeded or unfulfilled. 
The HSC results are subject to a lot of scrutiny. The school publishes some summary information at the time the results are released; more data is published in the Annual Report which is submitted by the end of June each year. I encourage people who are interested in our results to look at this data.  

However, it is staff who spend the most time poring over and analysing the results, exploring all the methods outlined above, and more. The data is explored subject by subject, class by class, student by student, and compared to previous years and published State norms. A report developed by both staff and an external consultant (who is used by nearly every comparable school in the State) is provided to the Board of ICL, the School's governors. 

Over the years in which I have been examining HSC results, I have become convinced that the best overall measure of our results emerges through the Grand School Average (GSA). This figure, established by our external consultant, is calculated on the relationship between the HSC result in every subject by every student and the ATARs that the students receive. This measure accounts for the relative scaling of subjects, recognising the merits of students attempting more challenging courses. 

The GSA at Inaburra has been rising since we began to measure it in 2008. This long-term trend is the best indication that we have that academic achievement at Inaburra is strengthening. Each student is being helped in his or her pursuit of excellence. 

Two final points:
Success is ultimately measured by individual students and their families in different ways. For some students, Band 6 results are a valid goal and object of celebration. Other students and their families, who may have walked a very different path in aptitude and circumstance, may well rejoice with different numbers. Many of our families also measure success more widely than the results in the HSC. 

In reflecting on the HSC and ATAR, I do not resile for a second in my conviction that a strong ATAR, by itself, is not an adequate outcome of secondary schooling. We need to be aiming for ATAR plus the character, capabilities and capacities of the Inaburra Learner Profile if our children are to be equipped to survive and thrive in the world that lies ahead of them. At Inaburra, we will continue to work hard to help our students to achieve strong results, but we recognise that the long-term benefits of their education will not be expressed in numbers.

No comments:

Post a Comment