1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. Hi Guest, welcome to the TES Community!

    Connect with like-minded professionals and have your say on the issues that matter to you.

    Don't forget to look at the how to guide.

    Dismiss Notice

ALIS reliability

Discussion in 'Heads of department' started by RobMathers, Nov 10, 2017.

  1. RobMathers

    RobMathers New commenter

    Hi All,

    I am Head of Department for Maths at a school with predominantly (over 99%) international students. We ask our students to sit the ALIS test on arrival and Senior Leadership set departmental objectives based on these. While I am fairly new to this role, I have taught here for a few years and know for a fact that for us, there is almost no correlation between ALIS and exam results at the end of the year.

    I wanted to know how accurate this data is for other schools/sixth form colleges with predominantly British students. I do not want our objectives pegged against something unreliable but I don't know how strong an argument I have at this moment in time.

  2. Skeoch

    Skeoch Lead commenter

    I'd start by doing some work on your school's data, not only for Maths but also for other subjects. What exactly is the correlation value?
    Then I'd contact CEM Centre at Durham and ask for their comments. They may well be able to give you some helpful datasets.
  3. RobMathers

    RobMathers New commenter

    That's very helpful, thank you.

    We have a data manager in school so I shall try and get this information from her. I hadn't considered contacting CEM; presumably they are interested in how accurate their data is for International Students too.
  4. RobMathers

    RobMathers New commenter

    Turns out the PMCC between the actual result and ALIS is 0.301 for the department.
  5. frustum

    frustum Lead commenter

    Can you write something you think might give a better indication, and administer it to your current cohort ASAP, so that at the end of the year you can say "look, this would give a better idea."
    Or can you look at which items on the ALIS test do seem to correlate better?

    Of course, if they want to set departmental targets, they're probably more interested in overall levels than which students are likely to do how well, but at least if you can get something that is a better indicator up and running, after a couple of years they'll be able to look at whether each cohort is stronger or weaker than the previous one.
  6. Skeoch

    Skeoch Lead commenter

    Our approach in a small school (might become unwieldy in a big one) is to use:
    • ALIS predictions;
    • Our historical data on how ALIS predictions pan out for our school - say the Klingon department exceed ALIS predictions by half a grade on average, but the Space Navigation department is generally half a grade below;
    • Individual knowledge of students, understanding a bit about their background and approach to life;
    • Internal exam results.
    These can then lead to reasonably helpful individualised predictions and targets, which can be adjusted as we know more.
  7. wanet

    wanet Star commenter

    I used to use the ALIS GCSE and their test predicions - generally confirmed what I knew - Low GCSE prediction but high test prediction - lazy, other way around - worked really hard for GCSEs- generally they didn't change when it came to A Level.

    And their predictions are really for cohorts - not individuals - unless it is a large sixth form then comparing depts from one years results isn't valid, however if several years show the same rends then they start to be valid.
  8. Skeoch

    Skeoch Lead commenter

    Agree with Wanet. We use these predictions to look at individuals rather than departments (or sets within departments); HoDs are therefore not held to account over cohort data, but rather over individual candidates - has John Smith done at least as well as we'd expect, given his various test scores, GCSEs, and other information? If not, why not, and are there lessons to be learned? Has Sally Jones done better than we'd expect? If so, why, and are there lessons to be learned? Of course the answer to the question about lessons to be learned will often be No, because John or Sally are both individuals with their own individual strengths and weaknesses.
  9. RobMathers

    RobMathers New commenter

    I gave all the new AS students a paper with A and A* IGCSE questions to help set them as we did not have the ALIS data until a few weeks ago. This result and ALIS were also not particularly strongly correlated. Pearson released a baseline test a few weeks ago through ActiveTeach/ActiveLearn which turns out to be similar to what we made, so it should yield some reliable data.
  10. RobMathers

    RobMathers New commenter

    That seems much more reasonable. It makes sense to me to use it on an individual basis rather than departmental.

    Looking at the data for our students though, it does seem unreliable enough that I would most probably ignore it when doing the results analysis. The impression I get from the comments is that most other HoDs would not do this.
  11. roydenkeith

    roydenkeith New commenter

    ALIS data is based on the past performance of students in your subject with similar GCSE results or their performance in a cognitive ability test.
    The problem is the new linear specifications mean that at best there is only one years data to base their predictions on and in some subject no data. So I would be sceptical.
    Another thing that bothers me is the prediction is based on average GCSE results not the result in your subject.
    Lastly they use a regression equation to calculate the likely point score which might be highly accurate but the point score could be anywhere between two grades, so is the target grade then the one above or below and where do you draw the line. So yes you might use ALIS as a guide but I would be wary of using it or having it used to wholly judge performance of students or teachers
  12. adamcreen

    adamcreen Occasional commenter

    Rob, I'm a Head of Maths. We have 70 students doing A Level Maths a year. The majority have GCSE grades 7 and 8 with some 9s. All were given a school target of A or B (produced by computer, whether FFT or ALIS or whatever). The ones who had a computer target of D or C were moved up to B by management. So I have to get 100% A-B whatever happens. There is NEVER any correlation between GCSE score and A Level score. Try to ignore any targets your school sets, difficult I know.
  13. binaryhex

    binaryhex Lead commenter

    @adamcreen What a joke. I worked in a school like that once, with silly targets that you were then expected to agree to, work towards, track, justify, set up intervention plans for etc etc. And people wonder why teachers’ workload is out of control, when you have totally stupid and unreasonable SLT dreaming up top grade targets for everyone on the loosest of evidence, which teachers then have to run around justifying and producing evidence for. I’d make it very clear that they are unreasonable and suggest what is reasonable, and work towards them. You have my sympathies.
  14. adamcreen

    adamcreen Occasional commenter

    Our aim is to get every student the best grade they can. We do intervention, we track, we test them every fortnight, and quite right too. But if I have a GCSE grade 6 student who will struggle to get an E at A Level and their target is moved to a B because D would have been part of a culture of low expectations, I won't worry if they get a D.

Share This Page