1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. Hi Guest, welcome to the TES Community!

    Connect with like-minded education professionals and have your say on the issues that matter to you.

    Don't forget to look at the how to guide.

    Dismiss Notice


Discussion in 'Education news' started by hermitcrabbe, Oct 20, 2015.

  1. hermitcrabbe

    hermitcrabbe Established commenter

    There are many things I could say about ALIS . Not least of all how the hell do Durham University come up with a " predicted" grade C at A level in a largely language rich subject ( writing essays, no coursework , all exam based) for kids with little or no English and low vocabulary skills ( usually ESL) ....yes I know the statistics ,but that's not my problem - except it is my problem. I see Vince Ulum has a similar thread on FFT.

    However, I would value a bit of help here.

    My school uses ALIS as a predictor and as a VA. Of course when kids are "predicted" C and get U ( which they will do as they do not speak English and cannot read the questions and I could tell you at the outset they would get U but they are left in my class anyway) I have a problem with VA. I have a document right now where I have to say how I am going to achieve the CEM predictions and improve VA in my subject.

    I would love it and be really grateful if someone could tell me how - one of you lovely teachers who can and has done it maybe care to share your good practice? I will also be PM'd on this.

    drek likes this.
  2. Vince_Ulam

    Vince_Ulam Star commenter

    It's insane to expect teachers to pull double time to fulfil the prophecies of some dumb expert system. Frankly you'd get more accurate predictions from a magic 8-ball than those which ALIS utters.
  3. lanokia

    lanokia Star commenter


    Vince, you do see the irony of mentioning a magic 8-ball when that is now the new standard of government measurement ... in the form of Progress 8!

    The irony will be lost on the government of course [not that they care]
    drek likes this.
  4. MrMedia

    MrMedia Star commenter

    I have written before and so have many others about the misuse of statistics.

    At large group level n=statistically significant number, these data sets do, often, help flag up where a large group is persistently underperforming.

    At individual level, these statistics are meaningless and dangerous at best for both student and teacher.

    For example, take 11 year olds running 400 metre times. At the age of 18 you might have a set of expectations about improvement. Given a thousand students at a time you might say one school is better at making improvements over another using statistics. However, the moment n=non-statistically significant number this data is clearly inappropriate for target setting.

    Student A receives private coaching at a local club and has been fast tracked into the senior level.
    Student B was bullied at school and took comfort in food and developed game playing alone as a coping strategy.

    Both students achieved the same time at 11 years old.

    What is an appropriate 400m target for Student A? Ditto Student B?

    Now I have gone to the two extremes of a data set, but you only have to work along the spectrum to find ESL, weak conscientiousness and all the other factors which affect students.

    Accurate and effective target setting needs to be challenging but achievable to motivate a student. Targets should not be used as a measure of the teaching or perceived 'progress'.

    I imagine Scotland will be paying for an appeal to the exam board for that penalty in the last minute! ;-) Should we sack their coach?
  5. hermitcrabbe

    hermitcrabbe Established commenter

    I think even I know about the statistical inaccuracies but can anyone tell me what I can put on the development plan? Strategies for achieving ALIS predictions and improving my VA against those?
  6. drek

    drek Star commenter

    Intervention, levelled revision guides, keyword glossaries, one to one feedback discussions and lots of mini assessments to check progress, school policies etc;
    Things you probably already do......
    You can prove you took the learning to the student but you can't and should not have to prove anything else.
    Don't for heaven's sake put the colour of pen you will be using to mark and give feedback :)
    Vince_Ulam likes this.
  7. JaquesJaquesLiverot

    JaquesJaquesLiverot Established commenter

    Does it also depend on your subject? We used to find that, with ICT, our students used to get an average of 1.5 grades higher than their ALIS predictions - we put that down to it being taught badly elsewhere.
  8. drek

    drek Star commenter

    I think that says it all :)
  9. hermitcrabbe

    hermitcrabbe Established commenter

    Ah, thankyou. Thats my plan completed then. I can send it in now :)
  10. hermitcrabbe

    hermitcrabbe Established commenter

    You know, if I weren't a bloke (stiff upper lip and all that) I would cry at this remark. Its deeply hurtful. Offensive even.
    The ICT dept in my school would refuse to teach the students that get pushed into my course.

    .... and therein lies at least part of the problem.

    The other part being that if you look at the individual performances and grades, generally none of those who I have in my classes do better in other subjects unless its art, photography or ICT (occasionally, if they havent managed to ship them off) where the LSA or technicians admit doing it for them and often, when its maths and science, they do far worse.
  11. JaquesJaquesLiverot

    JaquesJaquesLiverot Established commenter

    What would be your interpretation? It's all statistical, so if, on average, students in other schools with the same GCSE results are getting results 1.5 grades lower in the same subject, then either we were holding them back at GCSE (so that they were more able than their results suggested), or, somehow, we were adding more value at A level. Or the predictions were wrong for everyone, of course.

    If your students are so bad that a department would refuse to teach them, then presumably their predictions are quite low?
  12. hermitcrabbe

    hermitcrabbe Established commenter

    ALIS sets the predictions at a C (A Level). Many of them are EAL students with little or no English and a number of them have not even taken GCSE, so predictions, estimates, whatever you call them are not made from that. Neither it would seem is any account taken of verbal reasoning skills as usually these are quite low (as measured by ALIS data).

    IELT scores are around level 4/5 , if you are familiar with that.

    Those who have taken GCSE usually have a grade predicted from GCSE results of a U or at best an E (despite the ALIS grade).

    I am sensitive right now. Last years results were very bad. I have never had such a poor set of results. I am being asked to justify them and I am being blamed.

    I will notgive the whole cohort but just a few examples.

    a) male, had GCSE prediction for A level of a U. Had an IELTS in English of grade 4. Failed AS with just 10 accrued marks on the two papers yet was still in my class and I was expected to ensure he obtained a C because that is what ALIS predicited.

    b) female, predicted a U from GCSE. Had a grade G GCSE English. ALIS prediction was a D

    c) Male. No GCSE,educated in Italy. No maths and no English skills beyond rudimentary. Had a pupil passport for unreadable handwriting ( no extra time) and "slow processing" . Obtained a D in art (someone did his coursework). Got a U in my subject. ALIS prediction a C.
  13. hermitcrabbe

    hermitcrabbe Established commenter

    By the way, all of the above mentioned got grade A or B in their first languages and two of them got B in their second languages ( English was not even a second language for them).

    Their ALIS grades in all cases were C's.

    We have a fantastic MFL dept. They get wonderful grades, especially in languages we do not actually teach most times. - On average 1.5 grades higher than predictions and very opten better than most similar schools locally.
  14. JaquesJaquesLiverot

    JaquesJaquesLiverot Established commenter

    It sounds like the ALIS grades are wrong in your cases. The danger of statistical methods is that where you have small numbers of students with unusual circumstances you'll get erratic results. Maybe it's the case that most students with such poor GCSE results were I'll, for example, and recovered in time for their A levels.

    You'd think that common sense would prevail, though, and your SLT would realise that someone who got a G at GCSE isn't going to get a C at A level.
  15. lizziescat

    lizziescat Star commenter

    ............ common sense ....................... SLT............ ?

    The statistics are always right
  16. hermitcrabbe

    hermitcrabbe Established commenter

    Forgive me for being somewhat stupid,but when someone gives me a paper with my " Value Added" and residual scores on it and they amount to a big RED ( for it is colour coded) - 10 I have to believe that someone somewhere ( the HT and his DT&L?) believe that these predictions have some merit and I am failing.

    I have no idea how to change this unless I remove the students who cannot reach the ALIS predicted grades. Yes, I have done, am doing, the revision books, model answers, tests, marking with feedback but at the end of the day they have to go in a room and sit an exam, written in English, where they have to write in English and they have to write five long essays in English.There is no time for slow processing or failing to understand the question. I cannot go and sit the paper for them.

    I do not know where ALIS get their statistics from. I understand what is done and I understand how they arrive at their figures allegedly but grade C is not correct for students with rudimentary English and SEN in a language rich subject and no coursework.

    I know other teachers " talk" the students into leaving ( well, they do not talk to them, they are as far as I can make out plain horrible to them and make their lives hell until they choose to go). Those of us "stuck" with them because once they drop a subject or two the DT&L refuses to let them leave any more , end up with bad VA scores - we are set up to fail ( and then blamed for it).

    I do not give four x's what ALIS think their statistics are for , or whether they are being misused. This is a reality in most schools ( I think) in the country. Made worse by the fact that Durham University must be selling their product based on this kind of predictive power ...... otherwise why do schools keep buying it? What it really does , according to those experts here, is nothing like what it is being used for.

    Durham University ( and others like them FFT, ALPS etc. ) need a kick where it hurts.

    My rant over for this morning. I now have to go and deal with some "school sexism" where crabette is concerned.

    Have a nice day everyone.
  17. drek

    drek Star commenter

    We had some stranger walk into school last month claiming they were an 'expert' alas no one could make out at what, after sharing her observation and student questioning techniques!
    In her expert opinion after riding roughshod through the school, she opined that EAL gcse students with SEN, should be able to get a C.
    In essence there should be no bottom sets then in any school.
    Just 3 sets of students C b and a?
    Her background?
    Teaching in private international schools
    Perhaps she is one of those on the Alis team who confuse having a foriegn accent with being EAL and or SEN? Idiot.
  18. hermitcrabbe

    hermitcrabbe Established commenter

    Even International schools (there are some near me) cannot get such students grade C GCSE..

    They use the IGCSE anyway and do not enter all their students for this.They enter most for IELTS.

    My class is often made up of some students who have GCSE grade D/G and are re sitting with us.

    Many such then come into my sixth form classes at my school. Their average IELTS is around 5
    (rudimentary - can talk and write about themselves and their personal experiences, family etc. Can understand basic English). University requires IELTS of at least 7.5 ( so you have some idea).

    Its quite telling that many of these same students will take A level in their own language ( which they should be fluent) and still only achieve a B or even a C ( not an A).

    Your expert was spouting ALIS - which by the way does make a "prediction" .... its written on the top of the page for every student in the cohort who takes it. "Predicted Grade" it says......
  19. pinot

    pinot New commenter

    Because yoir school are using ALIS incorrectly that's why. On the spreadsheet the school Is sent there is a row for adjustment, you are meant to adjust this to reflect your school, so for example if your school over a few years say has an average differential of -0.8 then this is supposed to be entered to give an accurate prediction for YOUR school and so you can compare year on year. It is not for value added but to give accurate predictions so CEM offer the data for the country and subject, your school is meant to then adjust it based on historical mean differential.
  20. drek

    drek Star commenter

    When will the idiocy of running schools similar to chain store consortiums hit executive boards?
    Students and teachers are human. And the numbers are much smaller than the larger numbers required to make predictions on globally resourced inanimate objects.
    Durham Unniversity's intake sounds like they might be middle class or higher, which may be why the predictions work for their feeder schools and colleges.
    And why it would crash in schools with larger than avg SEN, EAL and FSMs.
    One school wanted to show it was doing 'extra' stuff for premium students, so it took them off time-table. Shoved them in a room with revision guides and a non specialist. Called it premium style intervention! I don't think the remit for making predictions (aspirational) is the same one for payscale judgements
    (finger pointing based on aspirational judgements).

Share This Page