1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. Hi Guest, welcome to the TES Community!

    Connect with like-minded education professionals and have your say on the issues that matter to you.

    Don't forget to look at the how to guide.

    Dismiss Notice

Quality consultation on Tes resources

Discussion in 'Tes Authors' Group' started by tesAuthorTeam, Jan 4, 2018.

  1. charliep86

    charliep86 New commenter

    Hello authors! Thank you all for the comments, feedback and ideas. It’s great to hear more from all of you, we encourage you to continue sharing your thoughts with us! It’s what helps us ensure that we build the right things for you, and make Tes better for everybody.

    There’s been a range of ideas discussed, which seem to fall across a few key themes, as outlined below - it’s a bit on the long side, so please bear with me!

    How to define quality

    We recognise that defining precisely how to measure the quality of a resource is highly subjective, and have had many debates internally and with teachers about this! The current system we use for determining resource quality (the five-star rating system) doesn’t contain any of this nuance, and so we will be reviewing this the near future to help us arrive at a measure that is more relevant to teachers using Tes.

    What kinds of alternatives do you think would work better than the five star system for measuring the quality of resources? We’d love to hear more from everyone on what you think makes a good quality resource, whether free or for sale, and how we might measure this better.

    How to encourage more reviews

    Ratings and reviews are super important. We have been exploring ways to help drive up the number of people rating and reviewing resources on Tes, and have seen some great successes already (particularly with reminder notifications), but have a long way to go. The idea of incentivising reviews is a great one - and certainly one that’s been tried and tested elsewhere - and is something we will likely trial.

    Are there other ways you can think of that would encourage teachers to come and review the resources they’ve used?

    Author and resource validation/approval

    Some of you have suggested that perhaps we need an approval or quality-assurance process for authors or resources - either for new authors as they join Tes, or for any resources that get published. This would definitely go a long way to helping solve the issue of quality, ensuring that only the best resources end up on the site. However, as many of you have noted, the overriding issue is that this would be incredibly difficult to manage and would require a great deal of resource to oversee.

    We have previously discussed whether there might be members of the community who could play a role of this nature, whether in reviewing new authors and resources, or moderating content (e.g. flagging spam, copyright or inappropriate content). We think there’s a huge amount of potential here, so it’s definitely something we will be looking to explore more in future. For example, there might be groups of people willing to review new resources and authors, which could help the community massively, or to help take down spam.

    Author badges is something else that has come up in discussions in this thread, and something that we have discussed here at Tes too. It’s definitely an idea we would like to explore more and run tests on in future.

    How do you think we might add a level of quality-assurance to the resources on Tes, without it taking up too much time and effort?

    Removing resources

    This is a hotly contested option - and one that we’ve debated numerous times here. Frankly, we can see the merits on both sides of the argument, but ultimately are reluctant to delete somebody else’s resources! If we did ever decide to pursue this idea, we would of course take appropriate precautions.

    We made some adjustments to the search algorithm a while back to help boost resources that have had more recent engagement, rather than those that are old and unused, so this has helped to mitigate the issue of old, irrelevant or unused resources. But, as many of you have rightly said, old resources still have their uses, and aren’t necessarily irrelevant just because they are old.

    How might we encourage authors to take responsibility for keeping their libraries of resources on Tes updated and relevant?

    Author royalties

    As many of you have previously remarked, the current royalty system is set up to favour volume of resources over quality. There have been a number of suggestions in this thread of alternative ways to calculate author royalties (from a paid membership model, to an advertising revenue share model), and we’d love to hear more suggestions around this from you all.

    We want to make sure that authors are being appropriately rewarded for their hard work, and that our system helps promote the right behaviours, by focusing on quality, not on quantity. So, please contribute more to this discussion too!

    What kinds of different models might we consider for calculating author royalty, that aren’t tied to the volume of resources uploaded?

    Other suggestions

    There have been lots more other great suggestions - including ideas of more tools for authors to help them interact with the teachers who use their resources, and ways for teachers to request resources from authors.

    To help give greater visibility of all the many great ideas that get submitted (by you and the folks here at Tes), we’ve been talking about setting up a way for you to view all of these ideas. We’ll be looking into setting this up over the next couple of weeks, and would love to hear whether you have any thoughts on this too.

    If there are any specific points I’ve missed off that you’d like to hear more about, please let me know and I can provide our thoughts on them.

    Please keep on contributing to the discussion, and don’t forget to tune in to the live Q&A session this Thursday at 8pm.

    Thanks,

    Charlie :)
     
    LivelyLearning, LizMavor and mrajlong like this.
  2. thinky

    thinky Occasional commenter

    Newly uploaded resource =ranking 0

    Resource wishlisted = +5
    Resource purchased (less than £3 = +10) (£3-10 = + 15) (over £10 = + 20)
    Review left = +5 per star (-5 if two stars, -10 if one star)

    Resource viewed but not purchased or wishlisted = -0.5

    Resource reported for copyright infringement -50 across all author's rankings


    Obviously more factors can be added and the weightings would need better consideration but the principle of good practise leading to higher rankings should eventually lead to better quality being more visible... providing authors know what the expectations are (right now I haven't a clue).

    There would also need to be a level of trust that such a system wouldn't be used to falsely promote particular authors in a subject like say maths mastery :rolleyes:

    Given the animosity from a significant number of TES users the reporting of copyright infringements and the star ratings would need careful management (but presumably this is the case already). The reporting of copyright infringements should be much easier in any case to facilitate the type of community management that would help to make it effective.
     
    CurriculumForAutism likes this.
  3. thinky

    thinky Occasional commenter

    I'll also repeat the broken record that things like custom categories and options for buyers to easily contact sellers and the ability for sellers to send updates to previous buyers will all help too...
     
    Kazg1 and CurriculumForAutism like this.
  4. elder_cat

    elder_cat Established commenter

    From what I have seen so far in this thread, the general concensus seems to be that something needs to be be done to sort the wheat from the chaff. I have only recently begun to publish resources, and as such do not profess to have the experience of other OPs, so apologies if I am approaching this from the wrong angle. The suggestions put forward by @thinky would seem a good place to start.

    I have never used a 'Wishlist' in my life, and I suspect there may be a great many others who could say the same. OK, I kind of get the idea behind it for paid for resources, but if these resources are indeed viewed by teachers across the world, then you could have a situation where a resource gets a good score for having been added to numerous wishlists, but never actually gets downloaded and purchased. It would need an accurate and reliable way of being able to match downloaded resources to the wishlists of individuals, and I'm not sure how "do-able" that would be ?

    So a resource costing over £10, gets a higher score than a resource which is free. I can see how that might help towards addressing the 'royalties' aspect. But would it not mean that a higher price resource would be favoured over a free resource, in terms of score, regardless of which resource was of better quality ?

    I would imagine this would work, provided that that:

    a. The issue of getting downloaders to actually review the resource is fixed.
    b. There are effective mechanisms in place for authors to be able to address reviewers comments. (That's assuming the reviewer hasn't simply ticked how many stars, without actually making any comments).

    Not sure about whether there is a relationship between the number of times it's viewed, and the number of times it's downloaded, in terms of quality. There could be numerous instances where someone views a resource, then decides that it doesn't meet their specific needs. That doesn't mean the resource is not good, just that it didn't meet their specific needs. Outside of the obvious links to specific curriculum content, I don't see any way at present that authors can reasonably be expected to cater for all potential users' needs.

    I agree wholeheartedly with the sentiment behind the idea of improving quality of resources made available on TES. But I do think it needs a great deal of careful thought about how it is done.

    Perhaps there is a case for free and paid for resources being made available separately, rather than from the same start page, so we are at least comparing like with like.

    Some sort of 'drill down' mechanism needs to be in place, so resources can be categorised better than they are at present. That would make it easier for viewers to find resources that actually meet their needs. The text box currently in use at the moment, where you are allowed to add a description of your resource, should allow a lot more than it does right now, and when you click on the 'read more' link on the web page showing the resource, it shouldn't cut off large tracts of what you have written.

    I suspect that identifying and quantifying what is meant by 'quality' and its' relationship to 'value for money, and enlisting the support of all authors, is neither simple nor straightforward. But in the long term, addressing these issue should benefit both authors and end users.
     
    kibishipaul likes this.
  5. thinky

    thinky Occasional commenter

    I wasn't suggesting the factors or the weighting I used as an example would be the best solution - clearly it would take much thought.

    I was proposing an approach towards encouraging the various things already suggested - in this case that resources be ranked via a system that authors understand.

    Currently we have no idea as to how resources are weighted for searches and listings.

    The one thing I can see on TES is that unless a resource happens to be awarded a 'top pick', a review or several quick sales it drops off the radar very quickly.
     
  6. thinky

    thinky Occasional commenter

    I think so.

    Free resources can't reasonably be subject to the same types of 'quality control' as commercial ones.

    I'm sure there's a belief that the free resources somehow promote paid ones but I tend to think that people seeking free resources aren't interested in anything else (and the *** model would suggest so too).
     
  7. kibishipaul

    kibishipaul New commenter

    There's a sort button !!?? :D
     
    Kazg1 likes this.
  8. CurriculumForAutism

    CurriculumForAutism Occasional commenter

    THIS^^
     
  9. CurriculumForAutism

    CurriculumForAutism Occasional commenter

    Other marketplaces such as ebay don't assess products for quality before sellers can list them, or get other sellers to assess validate them first, because it's up to buyers whether they want to purchase an item or not. The 'reward' for teacher authors is selling more resources, which will in turn encourage them to create more quality resources.

    I was emailed around October tyime and asked some specific questions about ways to improve the site, so I answered them as best I could, and no one from TES had the courtesy to get back to me.

    Instead of one gimmick after another (eg 90% off) in an attempt to attract new buyers, which only actually discourages buyers to pay full price for anything, TES needs to improve many features for both buyers and sellers.
     
    Kazg1, LivelyLearning and thinky like this.
  10. elder_cat

    elder_cat Established commenter

    My post wasn't meant as a criticism, sorry if it came across that way.

    I think you've hit the nail on the head there. The word 'algorithm' seems to get bandied about a bit, but I'm not sure many people have any real idea of exactly what criteria these 'algorithms' are actually based on ?

    Free resources benefits TES more than authors, as a way of attracting visitors to the site. Given that you have to be signed up to download anything, it's an efficient way of adding to the list of registered users. What that list of users gets used for, beyond distribution of resources, is another matter. I think the theory is that attracting more visitors, means more likelihood of a visitor buying something. Not convinced that actually works, hence the suggestion to separate 'free' from 'paid for'.
     
    thinky likes this.
  11. elder_cat

    elder_cat Established commenter

    But they do often have a 'reviews' section where purchasers can submit comments on the product and the service they receive.
    They may also have some sort of mechanism whereby a purchaser who is dissatisfied can complain, if they consider that the goods they have purchased are 'not as advertised', or 'not fit for purpose', and if eBay or Amazon agree with the buyer, then the seller is then expected to offer a refund. The other major difference is that with eBay and Amazon, the vast majority of the products on offer are physical items rather than the digital downloads offered on TES.
     
  12. thinky

    thinky Occasional commenter

    I think likewise.

    The large catalogue fits with the 'might is right' approach used to date. Maybe that works in terms of supporting revenue generate via Google ads etc. but I doubt it benefits an effective marketplace.

    I think it would be useful to know from TES what it is that they see as the priorities?
    1: a community of authors providing high quality resources for educators
    2: a simple selling/buying marketplace
    3: or a large userbase to which they can effectively sell their own products and/or those authors they commission/promote

    Personally I'd prefer it to be number 1 but it looks to me to be more like number 2 and leaning heavily toward number 3.

    If TES could be clear about their own ambitions it might encourage more relevant input into these types of discussions.
     
  13. elder_cat

    elder_cat Established commenter

    :)
     
  14. Maths_Shed

    Maths_Shed Occasional commenter

    That is kind of being done already with @MrBartonMaths 'resource of the week'. But who is paying attention to it? I had a resource of the week two years ago but it has still had only 700 views, I don't think it necessarily means it's one of the best 50 resources in a year but that's a very small percentage of UK maths teachers never mind globally who are aware of something that has been recommended in this way. I would have thought someone in every maths department should be looking at Resource of the week, every week. Evidently they don't and I don't think enough is being done to make teachers aware of quality resources. I've tried Facebook groups, Twitter etc* but feel TES seem to be sitting on their hands waiting for someone to look in and it's not working.

    *don't understand Pinterest
     
    CurriculumForAutism likes this.
  15. LesleyRitchie

    LesleyRitchie New commenter

    TES provides a useful resource base for users in far-flung parts of the world, e.g. Africa, so I would not like to see resources graded by price. I deliberately keep my prices low as, when converted from our currencies, they become quite pricey. I noticed that my sales dropped off when I changed the price of some resources from 2 to 3 quid.
     
  16. knapster

    knapster New commenter

    Hi. Thinky suggests a clear out - clear out the outdated and old rubbish that cutters up the current catalogue.
    I don't think that the amount of time a resource has spent on the site is necessarily an indication of its quality or usefulness. By all means, jettison resouces that pertain to previous GCSE specs but beyond that I don't think that the age of a resource should be a key driver of whether or not it should stay on the site.
     
  17. knapster

    knapster New commenter

    there are troll reviewers out there who are sticking 1 and 2 stars on resources for the sake of it without offering any kind of comment or constructive criticism.

    I wonder whether TES themselves incentivised low reviews some time ago by offering 'money back if disappointed' thing. Someone bought a 'Romeo and Juliet' lesson - left a one-star review with no comment and had their money returned. I removed the resource, felt some anxiety about why it had been regarded so poorly and asked the buyer for feedback to help me improve - didn't get a response, (which is fine; they're not my line manager after all) - but this experience was unique for me and I have quite a lot of resources on this site. I do wonder whether this person wanted to save time planning (fine) but issued the single star in order to bag a free lesson. The whole experience has left me with no more clarity about what would have constituted 'quality' for this buyer and a moderately bad taste in my mouth.
     
  18. thinky

    thinky Occasional commenter

    I wouldn't assume that just because a resource has been purchased in Africa that it's a poor school/teacher - there are plenty of wealthy schools across Africa.

    I provide resources to schools in Malawi and Uganda among others. I don't do it via TES because even if they could access them via a smartphone there's no way they could print them off in any useful way.
     
  19. JosieWhitehead

    JosieWhitehead Star commenter

    It's very rare, sadly, that anyone says thank you. I guess they appreciate them as they come in large numbers to use them. What a sad world!
     
    Alice K and thinky like this.
  20. PrimaryStarsEducation

    PrimaryStarsEducation New commenter

    Thank you for the session. It was very useful. I would like to see more of these. My thoughts:

    Royalty rates - I like different royalty rates (or would certainly not like them to go down). Would be great if they quality could determine what royalty you were on but I know this is subjective and difficult to measure.

    Feedback - Rather than us paying or funding it, TES could introduce a credit type scheme where customers earn credits for reviewing which can lead to a small reward. I do not like the idea of a teacher panel as I feel this would be more like a scrutiny, looking for errors.

    Communication - TES really need to allow buyers and sellers to communicate, whether this is by Q+A etc.
     
    Kazg1 and Resource_Creator like this.

Share This Page