1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. Hi Guest, welcome to the TES Community!

    Connect with like-minded education professionals and have your say on the issues that matter to you.

    Don't forget to look at the how to guide.

    Dismiss Notice

Author Consultation Follow-up 2: Reviews & Recommendations

Discussion in 'Tes Authors' Group' started by tesAuthorTeam, Jan 19, 2018.

  1. tesAuthorTeam

    tesAuthorTeam Administrator Staff Member TES Authors' forum host

    The purpose of this consultation is to work with the community and use your feedback to introduce ways to drive up resource quality, so that Tes can continue to be a trusted place for teachers to get their resources. For further detail and context please visit the original thread.

    Current review/rating system

    The current system we use for determining resource quality (the five-star rating / review system) doesn’t contain any of the nuance around whether a resource was useful, has met the needs of teachers, or in @thinky’s words provide “feedback towards actively improving a resource (instead it's a rate it good vs bad system)”.

    We will be reviewing this the near future to help us arrive at a measure that is more relevant to teachers using Tes. Some of the feedback we have had relates to promoting “further dialogue with customers / downloaders” on how a resource has been used, or whether they “recommend a resource”, as suggested by @elder_cat and @thinky. We really welcome these ideas and we will be exploring the possibilities here. We encourage you to share your ideas if you haven’t already.

    We ran an experiment last year to test out an alternative rating system, which allowed people to rate the resource based on a series of key criteria. We found that most people were willing to provide more feedback in this way. We will be continuing to explore different models that we can adopt here, and will be sharing examples of these with select groups in future, so please let us know if you’d like to be a part of this testing process.

    One of the other things we have noticed is that most people aren’t willing to provide detailed written comments or feedback, and many authors have told us that they really value these comments when they do get them, as it shows them that teachers are using the resources they’ve created. We want to explore more ways to encourage more teachers to provide more written comments and feedback on the resources they’ve used - for instance, to tell others how they used the resource in the classroom.

    What things drive you to leave more detailed written comments?


    Incentivising reviews

    Ratings and reviews are super important. We have been exploring ways to help drive up the number of people rating and reviewing resources on Tes, and have seen some great successes already, but have a long way to go. @lcallard21 and @mrsquenan have suggested that we send follow-up emails asking the person to review: the good news is that we already do this! We’ve had some great success with these so far, and are looking at ways to improve these notifications, and to find other ways to include notifications within the site experience, like making it part of the download process, as suggested by @elder_cat.

    Some feedback on the current system’s usability comes from @Krazikas: “Make it easier to provide a review technically instead of having to go 'all around the houses' to give a review”. Thankfully we have now addressed a lot of these issues with the most recent set of changes to the reviews system, and have hopefully fixed all the issues that were causing this to be unstable and not work reliably. We made some significant improvements to the usability of the reviews process last year with the introduction of the reviews page, though recognise that there are still issues that we need to address. If there are other specific pain points or issues that you have when it comes to leaving a rating and review, please let us know!

    An overwhelming response to this topic has been around introducing a mechanism to incentivise reviews and it has been raised by a great number of authors in this forum and outside of Tes, including @EC_Resources, @HSX, @alutwyche, @lcallard21, @Thedigitalstationer, @Krazikas. The idea of incentivising reviews is a great one - and certainly one that’s been tried and tested elsewhere - and is something we will likely trial.

    In terms of increasing the quality of the reviews left, @thinky suggests that “a rewards system for giving reviews it could give some weighting to how useful that feedback is...those reviewers that provide useful feedback would gain better rewards.” Similarly, this is an idea that many other sites use, to help gauge whether or not the person leaving the review is trusted, and helping the site understand whether the review itself was useful and trustworthy. We like the idea of rewarding reviewers that leave more useful feedback, and it’s definitely something we will explore further! We could also perhaps look at finding a way to establish “trusted” reviewers.


    Author ‘code’

    We feel there is a need to develop an ‘author code’ relating to behaviour on Tes around promotion of resources, soliciting of reviews and acceptable behavior which the community contributes to and agrees to uphold. We know the community cares about relevant and real reviews from teachers about the resources they download and purchase. This informs downloaders and buyers and helps teachers make decisions, and can make a difference not only to sales but to the quality of the resources themselves.


    We would like to work with you all - the author community on Tes - to write up a shared code of conduct that all authors can subscribe to, which helps guide how authors interact and engage with each other in these shared spaces and how authors conduct activities around promoting, as well as relating to the content they upload. We will be launching a separate consultation with you in the coming weeks to gather more specific feedback on establishing this. But, in the meantime, we’d love to hear more thoughts on this, so please contribute to the discussion!


    Tes Picks

    Many authors have asked for clarification on ‘Tes Picks’. It is worth noting that over the years the approach to Tes picks has undergone many significant changes, and as a result there may be some inconsistencies as things have developed and the library has grown. For clarification, we would like to reiterate that we do not perform a systematic review of all content on Tes, nor do we review resources on request, because of the sheer volume of resources published. We currently use an approach that focuses on an internal shift towards content hubs, topical and themed blogs, and other such projects, and view these selections as a Tes-equivalent of an ‘editor’s pick’.

    However, please be aware that this is not a systematic or full-time process, and in order to maintain quality, we can only focus on a few subjects / phases at a time. For insight into what the team look for, please check out this post. We appreciate your feedback about this and will include it as part of our ongoing review.

    There have been some suggestions around establishing groups of teachers who regularly review resources, whose reviews might have a more trusted status. We’ve discussed this idea at Tes, and would love to explore it more. Obviously there is a lot to unpick here, particularly around how the members of this group are selected, what process they go through, what their standards are, etc. We would be transparent with the community as we test out the ideas here, and would look to run things on a pilot basis, rather than establishing fully-fledged systems!
     
  2. tesAuthorTeam

    tesAuthorTeam Administrator Staff Member TES Authors' forum host

    Authors, on an additional note, if you have been misquoted or we've mentioned you in an incorrect context, please let us know and we'll fix it. Thanks
     
  3. thinky

    thinky Occasional commenter

    It's good to see the points being narrowed down.

    For the feedback concept to work there would need to be:
    - better, easier methods to communicate
    - clear examples that authors are willing to act on feedback and actively improve a resource (or feedback as to why they choose not to)

    Maybe authors who are willing to be proactive in this way could be invited to join a pilot involving something along the lines of invited teachers gaining access to their (specially discounted) resources on the basis they provide feedback not on how much they like a resource, but on how they would like it developed to work even better.

    This could maybe kickstart a culture shift. Alternatively it might be that this kind of proactive development is better as a distinct group (allowing for those who prefer to share their resources on a take it or leave it basis)?
     
  4. mrajlong

    mrajlong Established commenter

    I think this is a very urgent issue, especially with the 90% off codes being potentially misused. A review should be from a genuine buyer and not from a sympathetic colleague (who hasn't even used it). Reviews effect who is promoted and also can be misleading to buyers if they are not genuine assessments of the resource. It is so hard to get reviews and authors who solicit reviews in a more proactive (very diplomatic way of putting it) way are giving themselves an unfair advantage over other authors.

    This 'code' should really have been in place long ago, especially after the abuse of the voucher code system last year.

    Please, @tesAuthorTeam, put this in place sooner rather than later.
     
  5. thinky

    thinky Occasional commenter

    True. Although it would have more substance if TES themselves were part of that same code. I find the uneven promotion by TES far more frustrating than sellers attempts to promote themselves.
     
    mathsmutt likes this.
  6. ResourcesandCourses

    ResourcesandCourses New commenter

    Unfortunately I think that again TES have encouraged this, you can’t really blame new authors for this as they are just trying to get a foot on the ladder and get noticed. I admit wen I first started, I promoted these vouchers with my teaching peers as a way of trying to get some feedback on my resources, just to get me started, but having said that I know that they were used in the classroom. I‘Ve even given freebies to my peers to get feedback on my social media network. If Tes maybe had a section for new authors to help them promote their work, this might stop it happening as much. Also it might be good if they promoted different types of resources instead of the same types and subject. But at the end of the day it’s human nature, so,if there is a loop hole, people will find it and abuse it.
     
  7. mathsmutt

    mathsmutt Star commenter

    Sorry @Kazg1, I hadn't read this before I put my point into the "Royalties" thread.
    Don't stop at new authors!
     
    Kazg1 likes this.
  8. tesAuthorTeam

    tesAuthorTeam Administrator Staff Member TES Authors' forum host

    Thanks @Kazg1 for your comments so far. To be clear, we do absolutely encourage promotion of discounts and sales to colleagues and networks – we provide banners and marketing collateral to help support authors with this, and if you have any other ideas how we might support you more in this, please let us know! However, we do understand, as @mrajlong and others point out, there is perhaps some confusion in the community as to what is acceptable with regards to self-promotion and use of promotional mechanics. We agree there is a need to establish consensus on what this is. What do you feel is a fair approach to take and what would your preferred 'author code' look like?

    @thinky thanks for your feedback on the pilot program and your suggestions – really helpful. One of the challenges we foresee is recruiting people to such a program: what criteria should be applied and what would the parameters be, length of time it should run etc.? Would authors be willing to let appointed groups of teachers have free access to their paid resources for the purposes of reviewing/endorsing? We would love to hear ideas to further develop this.

    As mentioned in the 'Tes Picks' section above, we are hearing the comments about promotion raised by many authors and are taking these into account. As part of this, we encourage the community to share their ideas on what an alternative system or program would look like. As with some of our recent projects seeking exam revision resources*, we are working more closely with the community to meet the needs of teachers where demand is high. What else would you like to see?

    Please note: we are aware not all subjects and phases are covered here; this is due to limited capacity and audience size. These hubs are designed to be an ever-growing space. We have, in the first instance, included the subject areas listed based on typical site demand for Primary / GCSE content. As with our choice of filtered search, we will continue to review and adapt as necessary.
     
    Last edited: Jan 22, 2018
  9. mrajlong

    mrajlong Established commenter

    Getting all my colleagues/friends in school(s) to buy my resources using a voucher/code and then getting them to review them 5 stars is surely not acceptable. Although this practice has been allowed to take place with no repercussions to some. A "code" to stop this is worthless unless Tes act upon it when it happens. Tes must know when a resource has suddenly had a flurry of sales and then 5 star reviews from the same people (is there an algorithm for this?). It is unusual activity that should be flagged. When it happens occasionally I can understand Tes's difficulty to notice. When there are multiple sales and reviews for a selection of resources, especially from new accounts, then this should be flagged. As a buyer I would not believe these reviews anyway; it would deter me from buying the resource and make me doubt genuine reviews (I have witnesses this first-hand on Amazon when I believe reviews are fake). A "code" should definitely include rules regarding the solicitation of purchases/reviews. There definitely needs to be some way of verifying an account (maybe with bank details) so that you can only create one account per bank account - not many people have more than 1 or 2 - e.g. this bank account is already linked to another user blah, blah, blah. As for writing the "code", maybe others can think of a way of phrasing this...
     
  10. thinky

    thinky Occasional commenter

    One approach would be to allow an author a limited number of 'for review' vouchers each term. Let's say...

    - Author can distribute (say) 6 per term
    - Voucher enables a teacher to access a nominated resource for free (in return for feedback)
    - the feedback is marked as 'reviewer was provided with a free review copy'

    It would need to link with some sort of 'rate the reviewer' system too in order to demonstrate credibility (and for authors to mark down those that don't bother to leave a review). It could also link to the sort of rewards for reviews initiatives people have been suggesting.

    TES could potentially use this in promotion drives - the latest review copies available 2 weeks before each term etc. etc.

    The search algorithm could factor in a weighting in favour of resources with the highest ratings by the most trusted reviewers (a 'most trusted reviewer' would be in demand by authors and therefore gain access to more free resources for review each term).

    Obviously lots of detail to consider but I think something along these lines could provide benefits for all involved.
     
    MosaiK likes this.
  11. MosaiK

    MosaiK Occasional commenter

    Good ideas here to encourage more reviews @thinky, though I think these vouchers should be available electronically (and therefore coded) so that those of us who are not currently or no longer in a school and therefore lack the ability to freely distribute to colleagues. Electronically however, I think this could work as we could send review codes to colleagues we are most likely to still be in touch with.
    Erm... I think that makes sense. :rolleyes:
     
    Kazg1 likes this.
  12. ResourcesandCourses

    ResourcesandCourses New commenter

    I like the suggestion that someone previously made about being able to make your own vouchers/codes, this way authors could use them when they feel it's appropriate to do so. I like the promotional banners that you make, they are useful to have to help us promote on social media. To be fair, in my experience, I have found that Tes is one of the better online platforms that help us to self-promote as authors, a lot of the others, you have to pay a premium and then on top of that you have to buy or bid for advertising slots, which I think is outrageous. I don't see anything wrong with being able to give out voucher codes to encourage people to purchase.
     
  13. thinky

    thinky Occasional commenter

    Definitely - if it were me I'd be attempting to recruit reviewers from all sorts of places and as widely as possible.

    My wife would still have to pay for them of course :p
     
    Kazg1 likes this.
  14. nwilkin

    nwilkin Occasional commenter

    I don’t think users should be forced to leave a detailed review as it is likely to turn them off from leaving a review at all. I don’t mind the 4 and 5 star reviews with nothing written as I still am thankful that people have gone to the trouble of leaving that. It is annoying when they leave a 1 or 2 star review with nothing but I find if I ask TES they remove them anyway so it really isn’t a big issue for me.

    That said, I do think that if authors want more written feedback the reviewers could be given an incentive to write a more detailed comment, maybe set a limit of 50 characters so you don’t just get the “Great, thanks for sharing” review but something with a bit more substance.

    If they do leave a more detailed review it should give them a monetary reward maybe 20p off their next purchase, which could build up into a nice sum if they review lots of things. This way they are likely to review more products and would keep coming back to TES for resources as they have built up a decent discount. Winners all round!
     
  15. thinky

    thinky Occasional commenter

    However the system is tweaked, however reviewers are rewarded, however (even more) annoying popups/emails are used, people will mostly do the bare minimum needed to gain whatever reward is on offer unless they somehow feel like they care about making a contribution.

    'Feedback' suggests a resource is a work in progress and that people want to help improve it. But requires more work all round.

    'Review' is like having your work marked but with no interest in what happens next. But it's easy.

    Something worth raising is the question of what happens to the reviews already in place?
    • reviews for resources that were free (but are now paid) (this is very misleading)
    • reviews for paid resources in suspiciously close clusters (or by otherwise inactive accounts)
    As with the resources themselves - there's a lot of cleaning up needed for an impression of quality to be achieved.
     
  16. tesAuthorTeam

    tesAuthorTeam Administrator Staff Member TES Authors' forum host

    @thinky thanks for your further ideas on this, some really interesting stuff here for us to explore.

    To answer your questions regarding the below:

    We have received a lot of feedback in relation to this and it is something we are looking into at the moment. Watch this space.

    On the latter point, we do have checks in place regarding suspicious behaviour in relation to purchases and activity which doesn't look right or is potentially fraudulent. Some of the challenges which surround reviews are not unique to Tes but we do hope with the community's involvement we can work towards an agreeable route forward. As mentioned, a code of conduct relating to acceptable behaviour around promotion and soliciting of feedback which the community agrees with, adheres to, and helps us to monitor, feels like a sensible approach. Some have made suggestions already but we absolutely welcome more input on this.
     
    thinky likes this.
  17. MosaiK

    MosaiK Occasional commenter

    Might it be an idea to simply discount reviews from any kind of reward scheme or author level ladder? Having read what colleagues have written about the multiple possible pitfalls of reviews in this and other threads, I gather that there may not be a foolproof system to 1. encourage buyers to leave reviews and 2. avoid counter-productive reviews since it is difficult to check who is who and why some people think it funny to leave nasty reviews, sadly we live in a world where internet trolls exist.
    Another well-known website asks unhappy buyers to make indirect contact (through a dedicated resolution centre) with the seller before leaving a negative review to see if an issue cannot be resolved first. E.g. if someone used one of my resources and were unhappy with any aspect of it, say the pictures move about on the page, then I'd happily correct those as it would kind of make sense... and I'd send them a corrected version and maybe another one of my resources of their choice in exchange for either no review or at least not a negative one.
    I think that a right to reply from us to our reviewers would be good as long as such a reply somehow gets passed back in an email to the buyer - I believe some buyers may not frequent this site as often as some of us authors do.
    Last but by no means least, I have seen on resources this week that in return for a positive review, a seller offered a buyer a complimentary resource of their choice. What a great idea and brilliant incentive! I am considering doing the same. ;)
    Sorry this comment is rather long, but luckily for you my cat is demanding food now. :)
     
    thinky likes this.
  18. thinky

    thinky Occasional commenter

    This is a great suggestion.
     
    MosaiK likes this.
  19. mrajlong

    mrajlong Established commenter

    I am not sure how "ethical" soliciting positive reviews like this is. Wouldn't a buyer leave a positive review even if the resource wasn't that great just to get something for nothing? This would then be misleading to future buyers, especially if the author were ever to rescind the offer! I think any incentives to review need to be Tes lead...not through bribery!

    I think your points about being able have some kind of communication are bang on!
     
    MosaiK likes this.

Share This Page