Since its launch in February 2000, travelers and hoteliers have attempted to better understand just how accurate the reviews at TripAdvisor truly are.
Jonathan Barksy and Robert Honeycutt of Market Metrix, who's specialty happens to be hospitality feedback, compared 12-months of TripAdvisor reviews against their firm's proprietary index to find that – generally speaking – aggregate TripAdvisor review scores tend to be accurate.
The hotels used in this comparison were selected on the basis of their location, brand and type to achieve a diverse mix. Each hotel also must have received a sufficient number of responses. Statistical analysis was conducted to compare the similarity of scores for each hotel, the distribution of the data, and the change in scores.The most significant conclusion of this study is that the mean scores of hotels track very consistently and closely between TripAdvisor (CSI) and MMHI. This indicates that, when taken as a whole, the reviews for a particular hotel are a reliable measure of average customer satisfaction of that hotel, given adequate sample size. However, the variability of scores was found to be slightly greater among TripAdvisor hotels. TripAdvisor reviews were more spread out with more high scores and more low scores. While some persons may appreciate reading a wider range of reviews about a hotel, extreme observations may be distracting or even distort consumer perception.
(emphasis added above)
In other words, there are some reviews on TripAdvisor that – regardless of the hotel – simply aren't accurate. Individual reviews can skew negative or positive as a byproduct of an emotional experience. For example, receiving an amenity from the hotel to mark a special visit may elicit a more positive reaction on TripAdvisor than having a hotel misplace a guest's laundry.
On the whole, however, the median score for an individual hotel tends to be accurate and in line with established industry metrics for guest satisfaction.