1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. Hi Guest, welcome to the TES Community!

    Connect with like-minded professionals and have your say on the issues that matter to you.

    Don't forget to look at the how to guide.

    Dismiss Notice

ALIS reliability

Discussion in 'Heads of department' started by RobMathers, Nov 10, 2017.

Tags:
  1. RobMathers

    RobMathers New commenter

    Hi All,

    I am Head of Department for Maths at a school with predominantly (over 99%) international students. We ask our students to sit the ALIS test on arrival and Senior Leadership set departmental objectives based on these. While I am fairly new to this role, I have taught here for a few years and know for a fact that for us, there is almost no correlation between ALIS and exam results at the end of the year.

    I wanted to know how accurate this data is for other schools/sixth form colleges with predominantly British students. I do not want our objectives pegged against something unreliable but I don't know how strong an argument I have at this moment in time.

    Thanks!
     
  2. Skeoch

    Skeoch Established commenter

    I'd start by doing some work on your school's data, not only for Maths but also for other subjects. What exactly is the correlation value?
    Then I'd contact CEM Centre at Durham and ask for their comments. They may well be able to give you some helpful datasets.
     
  3. RobMathers

    RobMathers New commenter

    That's very helpful, thank you.

    We have a data manager in school so I shall try and get this information from her. I hadn't considered contacting CEM; presumably they are interested in how accurate their data is for International Students too.
     
  4. RobMathers

    RobMathers New commenter

    Turns out the PMCC between the actual result and ALIS is 0.301 for the department.
     
  5. frustum

    frustum Lead commenter

    Can you write something you think might give a better indication, and administer it to your current cohort ASAP, so that at the end of the year you can say "look, this would give a better idea."
    Or can you look at which items on the ALIS test do seem to correlate better?

    Of course, if they want to set departmental targets, they're probably more interested in overall levels than which students are likely to do how well, but at least if you can get something that is a better indicator up and running, after a couple of years they'll be able to look at whether each cohort is stronger or weaker than the previous one.
     
  6. Skeoch

    Skeoch Established commenter

    Our approach in a small school (might become unwieldy in a big one) is to use:
    • ALIS predictions;
    • Our historical data on how ALIS predictions pan out for our school - say the Klingon department exceed ALIS predictions by half a grade on average, but the Space Navigation department is generally half a grade below;
    • Individual knowledge of students, understanding a bit about their background and approach to life;
    • Internal exam results.
    These can then lead to reasonably helpful individualised predictions and targets, which can be adjusted as we know more.
     
  7. wanet

    wanet Star commenter

    I used to use the ALIS GCSE and their test predicions - generally confirmed what I knew - Low GCSE prediction but high test prediction - lazy, other way around - worked really hard for GCSEs- generally they didn't change when it came to A Level.

    And their predictions are really for cohorts - not individuals - unless it is a large sixth form then comparing depts from one years results isn't valid, however if several years show the same rends then they start to be valid.
     
  8. Skeoch

    Skeoch Established commenter

    Agree with Wanet. We use these predictions to look at individuals rather than departments (or sets within departments); HoDs are therefore not held to account over cohort data, but rather over individual candidates - has John Smith done at least as well as we'd expect, given his various test scores, GCSEs, and other information? If not, why not, and are there lessons to be learned? Has Sally Jones done better than we'd expect? If so, why, and are there lessons to be learned? Of course the answer to the question about lessons to be learned will often be No, because John or Sally are both individuals with their own individual strengths and weaknesses.
     
  9. RobMathers

    RobMathers New commenter

    I gave all the new AS students a paper with A and A* IGCSE questions to help set them as we did not have the ALIS data until a few weeks ago. This result and ALIS were also not particularly strongly correlated. Pearson released a baseline test a few weeks ago through ActiveTeach/ActiveLearn which turns out to be similar to what we made, so it should yield some reliable data.
     
  10. RobMathers

    RobMathers New commenter

    That seems much more reasonable. It makes sense to me to use it on an individual basis rather than departmental.

    Looking at the data for our students though, it does seem unreliable enough that I would most probably ignore it when doing the results analysis. The impression I get from the comments is that most other HoDs would not do this.
     

Share This Page