1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. Hi Guest, welcome to the TES Community!

    Connect with like-minded professionals and have your say on the issues that matter to you.

    Don't forget to look at the how to guide.

    Dismiss Notice

How much faith can we have in OCR exam results?

Discussion in 'English' started by Yewsername, Oct 5, 2009.

  1. It was recently announced that several exam boards plan to replace human examiners with computers. The reasons are, of course, money, and the fact that they cannot get enough suitably qualified volunteers to act as examiners. Why is there a shortage? Simple. Because they treat them so badly.
    I kept a blog of my experiences marking for OCR last summer. Here it is:


    I finally get the information they require by changing one of my referees (who is on a research sabbatical and therefore has a foreign address). I telephone all of them to find out their postcodes. One does not know. I have to change my referee again or find out somehow, I eventually find out via the internet and complete the last pages of the form. It has taken hours.


    <font face="Times New Roman" size="3"> </font> From: NAA Application Completed &ldquo;Thank you for completing your application to become a GCE English literature Examiner for OCR.

    We will contact you again once we have received satisfactory information from your referees.

    If you would like to apply for another level, subject or post, please use the login function at http://www.examinerrecruitment.org and select create another application.

    If you have any questions about the status of you application please contact us, you can find our details on http://www.examinerrecruitment.org/contactdetails/default.htm<font face="Times New Roman" size="3">&rdquo;.</font>

    So I contact AQA by e-mail. They inform me that they have never received my application. I tell them that they sent me an acknowledgement e-mail and I forward a copy of it on to them. They tell me they have nothing to do with these acknowledgement e-mails. They are issued automatically by the NAA website, using the AQA e-mail address, but this does not mean that AQA has got my application! I ask them what an acknowledgement does mean, and what the point is of sending it if it does not mean they have got my application? They say they don&rsquo;t know.


    <font face="Times New Roman" size="3"> </font><font face="Times New Roman"> April 26th 2008. The paper application form for AQA examining board arrives in the post. It is many pages long and I am busy. Also feel fed up so I throw it in the bin.</font><font face="Times New Roman" size="3"> </font><font face="Times New Roman">19th May 2008. I get a letter asking me if I want to be an examiner for OCR in English Literature AS-level. They warn me that this is not an offer of scripts. I reply by e-mail saying yes. </font><font size="3">20th May 2008. I get an e-mail from OCR asking me to mark scripts. I agree conditionally, waiting to hear more details of the requirements. They tell me that I will have to go to an examiners&rsquo; meeting lasting two days, 31st May and 1st June. They don&rsquo;t say where it will be. They don&rsquo;t tell me what books the papers will relate to, so I research it on the internet. I come up with a list of sixteen texts. Sixteen? With two weeks&rsquo; notice? I e-mail OCR and they say, yes, that&rsquo;s the list. </font><font size="3">21st May 2008. Some opera tickets that we had applied for months ago arrive. They are for 1st June. I reckon that it will be difficult to get there if I am working in London that day. I ring the Board and ask what time the meeting ends. They say 4pm. I think it over and decide I can just about manage to do both. I will be exhausted, but it&rsquo;s feasible, narrowly. So I tell them I can accept the post of script marker. They give me no idea when to expect the scripts.</font><font face="Times New Roman">23rd May 11.30. I return home from shopping to find that in my absence Parcel Force has tried to deliver a parcel of OCR materials. As I was not there to sign for them, they have taken them back to the depot, which is several miles away. The card says that the next time they can deliver will be next Tuesday, the 27th May! They require at least 48 hours for redelivery and the Monday is a Bank Holiday. I can go and pick up the parcel at the depot if I like but not until Tuesday. Why can&rsquo;t I pick it up on Saturday? When I ring the telephone number on the card I get a recorded message telling me they need 48 hours notice to arrange a collection. So it is no use driving over there even on the Friday or Saturday. If they had used Royal Mail, the parcel could have been delivered to my local PO which is within a mile away. But they didn&rsquo;t.</font><font face="Times New Roman">13.30 Royal Mail delivers my letters. One is an A4 envelope from OCR with detailed instructions about script marking. I am busy so I don&rsquo;t read all seven pages of it until that evening. It tells me the examiners&rsquo; meeting is actually going to be in Cambridge. By now it is too late to ring the Board and change my mind. I start to feel ill.</font> <font face="Times New Roman">Saturday 24th May. Bank Holiday weekend. I have flu and it is pouring with rain. I study my sheaf of instructions and CD from OCR. If I want to book overnight accommodation for the examiners&rsquo; meeting, I&rsquo;m supposed to contact them six weeks in advance. There is now less than one week to go. I realize that there is no way that I will be able to get back from the meeting in time to go to the opera on Sunday evening.</font><font size="3"> I ring the telephone number of the accommodation office. The line seems to be faulty. I get the &ldquo;this number is not recognized&rdquo; message several times. Luckily they have given me two other numbers. I ring the second one. I get the same well-modulated female voice telling me that this number is not recognized either. So I ring the third. The response is exactly the same. None of the numbers work. I wonder if it may be something to do with the weekend. You can&rsquo;t expect people to be working on the Saturday of a Bank Holiday weekend. I will just have to wait until Tuesday morning.</font><font face="Times New Roman" size="3"> </font><font face="Times New Roman">27th May 2008. Tuesday morning. I wait near the front doorbell from 8.15 until 10.15 but no parcel van turns up. At 9.30 I ring Parcel Force and ask them if the parcel can be delivered to my nearest PO. They tell me they are not allowed to deliver exam papers to a PO. They have no idea when the van will arrive. It could be morning or afternoon. I just have to wait near the front door for the rest of the day if necessary.</font><font face="Times New Roman"> </font><font size="3">9.45am. I try to e-mail the board, and find that something has cut off my internet connection. It was working last night. I ring the OCR central office and a recorded voice answers. &ldquo;Thank you for calling OCR. If your call concerns Interchange, please press 1. If it concerns MS1 please press 2. If it concerns asset languages please press 3. If your call is about 14-19 year-old qualifications, please press 4. For all others, press 5.&rdquo; As most of my qualifications are more than 14-19 years old, I press 5 and I get piped music of a vaguely jaunty, light-jazzy kind. &ldquo;Thank you for holding. Your call will be dealt with as soon as possible.&rdquo; More piped music. Then a guy with an Asian voice answers. He tells me that Rosie is not available, and that the meeting is definitely in Cambridge. I tell him I could not ring the accommodation people directly, so they put me through to the switchboard. </font><font face="Times New Roman">Eventually they connect me with the office handling accommodation for the examiners&rsquo; meeting. They say there are no more rooms left in the college but they will try to get me booked into a hotel for the Saturday night. They ask me if I want to book for dinner in the hotel. I say yes, as I don&rsquo;t want to be wandering around looking for somewhere to get a bite to eat. They say they could leave it open for me in case I want to decide later. I say no, please just book, as I really don&rsquo;t want to have to make any more decisions or arrangements. Have I got any special requirements? No, I will eat anything. I can walk up and down stairs. I am not a dalek.</font><font size="3"> I spend another hour on the telephone checking all the coach and train times between Oxford and my destination before realizing that it will be impossible to get there on Saturday morning in time for a 10am meeting. So I ring OCR again. I get through the options a little faster now. &ldquo;Please continue to hold. (Piano music of a repetitive and pedagogic style.) An advisor will answer your call as soon as possible.&rdquo;</font><font size="3">The advisor materializes, and I ask, &ldquo;Can I travel to the meeting venue on the Friday and claim for two nights&rsquo; accommodation?&rdquo; No, he tells me I can&rsquo;t. Rosie is out of her office, but I can leave a message on her answer-phone, if I like. &ldquo;Goodbye. No problems. Take care.&rdquo; NO PROBLEMS? TAKE CARE OF WHAT???</font><font face="Times New Roman"> I realize that I have not sent them back a paper acceptance of the script marking, so I could still chicken out of this if I want to. I haven&rsquo;t signed a contract yet. </font><font face="Times New Roman">I leave Rosie a message.</font><font size="3">10.53 I am still sitting here near the doorbell waiting for the exam papers to turn up. Royal Mail arrives and delivers two letters.</font><font face="Times New Roman">No sign of Parcel Force. I pick up the phone and find there is a recorded message from a young lady at Parcel Force saying that their van will be coming to my house at about half-past one today. Well, that gives me two hours of freedom. I can go and have a shower and get dressed. I check the internet again and it is still down. I make a cup of coffee. </font><font size="3">11.21 BT rings and asks me the telephone numbers that I could not get through to. I hunt through my sheaf of papers and give them the numbers. I am told that none of them are residential, so I will have to be transferred through to business lines. I tell business lines the same three numbers. They inform me that all of them are defunct. &ldquo;Not a working line.&rdquo; No wonder I had difficulty ringing accommodation!</font><font size="3">11.35am. Rosie gets through to me on the phone. She tells me that if travelling on the Saturday morning would entail leaving before 6.30 am I am allowed to travel to the meeting venue on the Friday and claim for two nights&rsquo; accommodation. She checks the train times, and tells me yes, I am allowed to travel up on the Friday. She suggests that I ring the accommodation office. I confess to her that I have had difficulty getting through to their telephone numbers. Yes, all three of them. She puts me through. I tell them my examiner&rsquo;s number and say I now need accommodation for two nights, Friday and Saturday. They tell me there is only one problem with that. (ONLY ONE PROBLEM? WHOOPEE!) &ldquo;Accommodation here is like gold dust. Getting it, even six weeks in advance, can be difficult. But we&rsquo;ll see what we can do. We&rsquo;ll ring you back. OK?&rdquo; &ldquo;OK. Thanks.&rdquo; </font><font face="Times New Roman">11.45am. I ring the leader of my group. He tells me somebody dropped out owing to illness. I confess that out of the sixteen or so set books on the list, I only own half of them. Most of the modern ones I&rsquo;ve never read. He says that the details are on the website and I apologize for the fact that my internet connection is down. He gives me the exact details of which poems are selected and promises to help me get hold of some texts. </font><font size="3">12.23 The doorbell rings. It is Parcel Force. Luckily I had not assumed I could go away for two hours. They deliver an amazingly thin package, really just a large A4 letter wrapped in plastic. I sign for it and ask if there is anything else. He says no. </font><font face="Times New Roman"> I open it and find three answer books. I have waited in all morning for three scripts. There is no exam paper in with them. I ring the Board once again, and ask what has happened to the other two hundred scripts. They search through their records and confirm that more are due to arrive but have not been sent to me directly. They have been dispatched to the Board, which will be sending them on. Some may not arrive until the end of next week. So my life must remain frozen in expectation of the advent of a Parcel Force delivery van at any hour of any day. </font><font face="Times New Roman"> I really need access to the website so I stay up late fixing up a dial-up connection to the internet. I try first with my usual computer, but find that since installing a cable connection, the old modem inside it seems to have been disabled. So I start all over again using my old, very slow laptop. It finally works at 1 am. </font><font face="Times New Roman">28th May 2008. I start hovering near the front door at 8am but no parcel turns up, except a spark gun I bought on eBay before my broadband connection failed. I get a letter saying that I am staying in a different hotel on each night of the weekend meeting. That is the best they can arrange. I spend at least an hour on-line struggling to buy the required texts with my doddery old computer.</font><font size="3">29th May. I ring National Express, &ldquo;Official transport providers for Wembley Stadium,&rdquo; and try to book my seat. They tell me that a paper ticket will be sent by e-mail and must be printed out and shown to the coach driver. Unfortunately my printer is connected to my better computer, which cannot get internet. My old computer, which has dial-up, is not connected to a printer. After consulting his manager, the chap on the line gives me a number to show to the driver. He says that will be all right. I go and try to download the e-mail and it has not arrived anyway. </font><font size="3">30th June. Having cancelled the pupils I would usually teach on 31st June, a Saturday morning, (which represents a loss of income) I catch an evening coach to my destination and arrive at half-past ten. It is nearly 11pm when I check in to my hotel. I spend my time en route revising the course-texts so that I know them really well. For some reason this does not give me a headache, or make me travel-sick. I am winning. </font><font size="3">31st June. I pack my suitcase at 9am (after eschewing most of the vast unnecessary hotel breakfast, aimed at the grossly obese) and check out. The meeting starts at 10am so that people who need to drive over this morning have a chance to get here. It is in a comfortable hotel conference room and everybody has been given a pad of paper and a free biro saying &ldquo;Made of recycled plastic&rdquo;. I meet my team leader. From 10am till 11am somebody talks about old examiners and general reminiscence, until it is time for coffee and biscuits. When we resume, at 11.30, we finally start to mark photo-copied scripts, following the marking guidelines, then compare our results. This is the standardisation process. My grades are not too far off but each script takes me about half an hour so after only three, it is time for lunch. Lunch is a very generous affair, with plenty of choice, plenty of food and plenty of time. Huge creamy confections and trifles are displayed on a buffet. I settle for a fruit salad. </font><font size="3">In the afternoon I confer with my team leader and mark five more photocopied scripts. We are told to mark seven REAL papers that evening. Problem &ndash; I&rsquo;ve only got three. Somebody kindly arranges to lend me some of hers, a pack of four. I take these back to my new hotel at 4pm when the meeting ends and mark them sitting on the bed. Then I find that I seem to have a total of only six. The pack which was supposed to contain four now contains only three. Have I lost a script? I wonder if my team-leader will shoot me. There is loud music in the hotel until 11.30 pm and I make two complaint calls to reception before it stops and I can get to sleep.</font><font face="Times New Roman">1st June. I tell my team-leader that I think I have lost a script. He says, &ldquo;Oh, Lord.&rdquo; We check various possibilities. We find the person who lent them to me. She looks in the pack and decides that there must have been only three in it. I feel somewhat relieved.</font><font size="3">We do some more standardisation. I am getting better. At 11 am it dawns on me that from now on, everyone is just marking their own scripts. I have none left. I wonder how early it is possible to leave without appearing rude. There is still one course-text which the team leader cannot find for me, and I will have to go on trying to buy it on the internet. I stay for lunch &ndash; the conversation is all about voluntary euthanasia schemes - and leave at 2pm. This means I can claim fewer hours&rsquo; pay but I prefer to go home. Why charge the board money for marking I am not able to do until I have some papers?</font><font face="Times New Roman">Monday 2nd June. I stay in all day expecting a Parcel Force van. At 4.30 a box of exam papers turns up - delivered by DHL. I am totally mystified, but sign for it. This is still not my full allocation. A hundred and thirty scripts are still expected. At 8pm, I slit open the box, and start marking. I follow the marking scheme piously.</font><font size="3">Wednesday 4th June. I send the first ten marked scripts to my team leader for his feedback. I am not supposed to do any more until he replies.</font><font size="3">Monday 9th June. My ten specimen scripts come back from my team-leader with some remarks. He says that my standard of marking is fine and gives me the go-ahead to mark the rest. Unfortunately the rest have not yet all turned up. I keep ringing the Board to ask what has happened to them and they confirm that some are still due to me. And the one text which is so difficult to get hold of is the one that seems to be most popular with candidates.</font><font size="3">At 4pm I find a card saying that the course-book I had ordered over the internet has not been delivered as I needed to be there to sign for it. It took a week to come as it is only published in the USA and there are no British suppliers. They cannot now deliver it again for another 48 hours as you have to book re-delivery. I start to wonder why the OCR can afford to pay for examiners to stay in posh hotels and have vast breakfasts, but does not provide the course-texts for us. Their priorities seem a bit wonky.</font><font size="3">I have to mark what answers I can without this text. I find the standards of work rather shoddy. Many candidates cannot spell or punctuate correctly, or even copy a spelling correctly from the text in front of them. Much of the handwriting is terrible and this slows down the job a lot.</font><font size="3">Wednesday 11th June. I wait in all day for the missing book. It is finally delivered at 4.30pm. I spend the evening reading it. I now have little time to mark the scripts before the deadline.</font><font size="3">Thursday 12th June. This is supposed to be the deadline for sending in 40% of my scripts.</font><font size="3">Monday 16th June. I send three bundles of marked scripts to OCR. There are still quite a lot of scripts that have not arrived! I make more frantic phone-calls.</font><font size="3">Tuesday 17th June. Parcel-Force tries to deliver something while I am out. (I am now teaching a summer-school part-time so I cannot stay at home all day, every day).They leave a card. I have to book re-delivery on-line. I am told that every time I do so I am entered in a competition to win a free laptop. However, I do not win. </font><font size="3">Monday 23rd June. My missing scripts finally turn up. They have been mislaid in an OCR office somewhere or other. I have roughly one week to mark them as the final deadline is 3rd July.</font><font face="Times New Roman">24th June- 2nd July. I spend several hours every day marking and get my scripts in by the deadline. I also spend hours filling in the complicated claim forms for payment and expenses. I claim for the purchase of all books I did not have, and wonder why the Board does not simply provide them instead of wasting our time and effort like this. A cheque arrives some time later for the full amount. With it comes a letter inviting me to re-apply for the next lot of exams and assuring me &ldquo;All of our assessors on GCE English Literature are highly valued and we very much appreciate the hard work you put in each year.&rdquo; It is signed Ian Cook.</font><font size="3">19th September 2008. I get a letter from OCR telling me that my marking was inaccurate and that I will not be asked to do any more. The letter states that &ldquo;feedback from assessors is not provided after the first sample during the marking period&rdquo;. Why should it be? If the first sample was approved, why is it necessary to go on re-assessing it? A long correspondence follows, during which I point out that my marking followed the guidelines of my team-leader, a senior academic who has been working for OCR for aeons. He was quite happy with it. They write back telling me that all my marking was bad, and that in one case I had under-marked a paper by 16%. They had to send all my scripts to another examiner to re-mark. I tell them that this was not the message I got from my team leader. If there is any disagreement, I am as well-qualified as anybody could be to judge such matters, and I certainly ought to have been consulted rather than just having my opinion dismissed. It is a fact that examiners often disagree, even at university level, (a friend of mine who teaches at Wadham College tells me the same thing happens when marking Finals papers). I ask you, my reader, having perused patiently my faithful account above of the chaotic and incompetence at the Board itself, who do you think in the case of disagreement was likely to be wrong? Did their performance inspire confidence? Hardly.</font><font size="3"> They do not accept my appeal, and there is no higher or external body that has the power to intervene. I conclude that the OCR board has no consistent standard for marking, and that they are merely eager to give generous grades to as many candidates as possible. I certainly do NOT feel that I am &ldquo;highly valued&rdquo; or that my work is &ldquo;appreciated&rdquo;. I feel that I have been kicked around and exploited by people whose own competence is very questionable.</font><font face="Times New Roman">November 2008. My daughter, who is now in upper sixth form, goes for the Oxbridge interview. It takes all day and involves an overnight stay. She has to sit two written tests and undergo three interviews! I ask her school why this is necessary. The teacher tells me that &ldquo;Universities these days have little faith in A-level results. They like to make their own choice.&rdquo; Her school is doing OCR. On reflection, I am not surprised at the attitude taken by the universities. Everybody in her class got an A for English AS-level!</font><font face="Times New Roman">April 2009. Another teacher at my daughter&rsquo;s school tells me that the English department is going to switch from OCR to the Welsh exam board, as they are so disillusioned with OCR. &ldquo;They seem to be in chaos. Nobody has any respect for their grades.&rdquo; I cannot pretend to be surprised. </font><font face="Times New Roman" size="3"> </font><font face="Times New Roman" size="3"> </font><font face="Times New Roman" size="3"> </font><font face="Times New Roman" size="3"> </font>
     
  2. I have to say my experience has been completely different. That doesn't mean I doubt what happened to you, just that I found OCR to be great as an examiner and teacher in Physics.

    I applied direct to OCR and received an offer to mark two different papers. As I didn't know how long it would take I only accepted one. Everything went fine with marking (delivered and completed online). I didn't have to attend any standardisation meeting but did have to complete standardisation scripts. Once into the live marking there were several times I disagreed with mark scheme or my team leader questioned my marking. I guess there was a pretty equal balance of ones I won and ones I lost - any time they strongly disagreed the reason was explained and I didn't feel trampled.

    From the teaching side I know of several students who were accepted at universities with lowered offers due to doing that option (OCR B). We no longer offer it as management decreed it was too hard and we were to boost results by changing to AQA!
     

Share This Page