Based on some of my recent posts about reviews I was contacted by ServiceMagic. They explained their process and quality control for reviews (they claim 500K consumer reviews). Here are some verbatim excerpts from the email I received:
Only homeowners who were matched to a contractor through ServiceMagic are allowed to submit a review, so the “legitimacy” issue is taken care of at the onset.
We have a team of 25 people who spend much of their time reviewing each R&R before it’s posted. We do this to identify any unsatisfied homeowner (rating score of three or lower). That score triggers customer service outreach to the parties involved. In roughly nine out of ten instances the negative rating is the result of a misunderstanding or a lack of communication between the homeowner and service professional. Consequently, our staff is trained to mediate and help the parties reach a resolution. In most cases, the consumer retracts their negative rating after the situation was rectified by the service professional and both parties are happy as a result.
We also give service professionals the opportunity to respond to ratings posted about them. We know there are two sides to every story, and do not wish to leave one side without a voice. That’s especially the case when a negative rating gets posted.
We’ve added the ability for consumer-reviewers to reach out to one another. Someone posting a review can indicate that they’re willing to be contacted by other consumers in the event they have a specific question about what’s been written, or in a general sense, the performance of the contractor they’ve been matched to.
Angie’s List does something similar with dispute resolution.
I especially like the final point about consumers contacting one another. That obviously doesn’t apply in a category like restaurants, but it’s very useful and valuable in “high consideration” categories such as remodeling and home repair.