Who Critiques The Critics?

Stewart Tongue
The review site market segment has grown rapidly in recent years and during a recent conversation, one paysite whose site I asked to review, said rhetorically: "I always wonder who critiques the critics." It's a fair question and while paysite owners and review site owners do have much in common, his comment brought into focus a perceived divide that many believe exists between them.

Some paysite owners and webmasters believe the entire review process creates a one-way dynamic where review site owners subjectively dole out grades and critiques for paysites that they know little about. Some review site owners complain that many paysite owners are too thin-skinned and are entirely unable to accept even the most constructive criticism. Webmasters claim that review sites are glorified link traders who gain from traffic leaks as a reward for positive reviews. Surfers on public boards often thank review sites for serving as their advocates in dealing with certain paysites. With a topic this juicy and passionate people on all sides of the issue, XBIZ decided to take a closer look at the critics and those who critique them.

In getting as realistic a cross-section of views as possible for this article, well over 50 contributors were contacted. Review site owners, webmasters and paysite owners familiar with the review process were all asked for their opinions. Somewhat surprisingly, while the pool of people contacted was far from being a scientific sample, there is at least circumstantial evidence that smaller program owners are leery of being quoted on the record when review sites are the topic at hand.

Some of the most vehement adversaries of review sites asked that their views be included in this discussion but were adamant that their names not be attached to their comments in any way. A few said they were concerned about retaliation by review site owners who they believed could "dish it out" but could not "take it." One particular interviewee went so far as to suggest there is a grand conspiracy among review site owners and angering the one he had a problem with might somehow cause his sites to be handled unfavorably when reviewed by any of the others.

Larger more established programs seemed to have a much more cerebral view of review sites overall. Mark of BlazingBucks said: "Review sites are excellent in terms of marketing a membership site; however, I say this based upon how well our sites do with review site traffic. There are a handful of very reputable review sites; however, like any other niche of sites, there are the good, the bad and the ugly."

Some suggested that review sites are a worthwhile tool for them to use when evaluating their own product line. Jean-Marie Kesch of HustlerCash Sales & Marketing said frankly: "We encourage independent reviews of our sites, as this gives us sometimes invaluable third party assessments of our offerings. We evaluate all reviews and analyze their arguments, good or bad and many times use this information to make changes and improvements on our end."

When asked specifically about the possibility of a "bad review," SilverCash Albert explained: "In our experience with review sites, we encourage constructive criticism to our operations because if we cannot embrace criticism, then feedback to improve our business will be taken as a personal attack and that certainly would not stimulate growth or creativity in competitive advantage."

Perhaps the willingness of top tier programs to speak their minds stems in part from the fact that their sites are likely to get very high scores from most review sites. Some smaller and newer paysite owners claim there is a bias by reviewers in favor of larger more established programs. Others believe the high scores are honest and impartial results caused by the simple fact that larger programs have bigger budgets which allow them to pursue better talent, more sophisticated equipment, high-end professional site designers, teams of full time employees and all of the other advantages that money and a lengthy track-record provide.

One paysite owner who has been undeniably successful and at least equally controversial is 12clicks, the owner of 12clicksCash. In his view, review sites are fundamentally flawed because they have goals that he feels differ from the paysite owners whose sites they review: "There's an implied bias against what is best for the program operators. Whoa, this site costs $39 when you can get the same thing from this site for $29. As a program owner I find that to be a nuisance." He went on to explain his views by saying: "The flaw in the review site model is in thinking that if the review site takes less money from the program owner in an attempt to give the surfer better service that: A) the surfer will actually know it, B) the surfer will care, and C) the surfer will remember and return out of thanks. None of those imaginary happenings are worth making less than possible from your site."

The presumed job of any paysite owner is to increase sales and profit margins, but is that really so different from what review site owners are trying to accomplish with their own internet publications, or are they just seeing the same ocean from a different shore? While paysite owners are tracking their sales and their retention, review site owners are attuned to other commodities like credibility and reputation.

Long established review sites like The Best Porn are notably successful from a financial perspective but as VegasKen explained: "The pure honesty of our reviews has gained us a lot of respect from our users. This has really been one of the key ingredients of our success. I also think that our honesty and integrity has earned us respect within the webmaster community as well. For every one webmaster who gets upset about a review, we have five that thank us for being honest, and letting them know where their growth opportunities are. Surfers will quickly realize if a review site is merely an advertising vehicle in disguise. You can just look at those sites and immediately see the few affiliate programs that they are pushing. This shortcut will work in the short term but user retention is likely to be low."

While there is no shortage of program-owned sites built with the specific purpose of offering biased 'pseudoreviews,' getting any of them to speak on the record was about as difficult as you might imagine it would be. Nobody wanted to be quoted as saying "yeah I built a mock review site just to give my own sites high scores and my competitors low scores as a method of driving traffic to my own sites."

That being said, it's important to differentiate between honest review sites giving a "bad review" and dishonest sites giving a "self-serving" review.

As to the idea of an honest review site giving a bad review, the owner of PlanetClimax, a PR5 review site that has been generating sales for years, had this to say: "Paysite owners shouldn't be upset about the review if the review is not what they expected it to be. After all, this is a competitive market they work in and they shouldn't forget that every person sees things differently. Review sites are just one way of monetizing their products and not the Alfa and Omega of porn sites where it's to be decided who lives and who dies... what review sites are basically is just one segment in the adult internet affiliate marketing playground that have a lot of content (read: search engine food) and they try to capture the more experienced porn consumers."

Research for this article included contact with many other review site owners of long established review sites like RabbitsReviews and HonestPornReviews, as well as my own experiences operating and discussions with promising newcomers like StrokeKing.

After looking through countless review sites with variations in their style, their design and in some cases in their agenda, I spoke with many long time industry veterans seeking a broad understanding of the review market segment. It was Sherm of who seemed to have the best grasp on the range of players and pretenders in the review market these days.

"Keep in mind that not all review sites are unbiased and act as a guide for the surfer," Sherm said. "Some are little more than keyword intense pages targeting search engine traffic and offering discount join rates. Just like other types of sites on the web offering free content, surfers are able to spot quality and are smart enough to move on to a more legit review site if they just see cheap attempts at getting a join.

Those legit review sites are the ones with the bookmarkers, the ones with the larger shares of the traffic, and the ones that will ultimately achieve the best ratios. Paysite owners should strive to have a good relationship with these reputable webmasters, making it profitable for both parties involved.

Sherm added that aside from just looking for great affiliates, paysite owners need to keep an eye on reviews and actually listen. Review sites look at endless numbers of paysites and they have an idea what features are industry standard. If a review site finds flaws in your paysite setup, do your best to fix these oversights and resubmit. Not only will the review site owners thank you by attaching a higher score to your site, but your surfers will most likely commend your hard work by recurring at a higher percentage than they would have previously. In these days of cluttered ad campaigns all over, consider review site owners as not only affiliates, but as consultants who offer great advice while pushing joins through along the way.

It's not a secret that many review site owners do not write their own reviews. Some do an excellent job by exercising strict editorial control to ensure the quality of all of the information that they are providing. Others seem happy to post just about anything on their sites in a blind quest to obtain more content as inexpensively as possible. If a reviewer is earnestly asking you questions about your site and is interested in providing accurate information, then perhaps the cooperative consulting impact Sherm speaks about is something worth considering.

When operating honestly, a review site is an independently owned and operated filter that is trying to assist quality paysites and knowledgeable surfers in finding each other. When one sees the numbers of joins and the consistently exceptional ratios that quality review sites produce, it seems hard to maintain an argument that review sites are seeking to reduce the profitability of paysites in general.

It also becomes clear that spending a few grand to buy biased or inaccurate reviews is not what building a quality review site requires. While artistic design, quantity of sites reviewed and search engine placement are all important parts of a review site's success, credibility and reputation seem to be the common denominator among the most successful ones in converting and retaining customers.

Paysite owners often withhold free access from review sites that they do not respect. Other review sites carefully decide whether or not to list a reviewer's link in their 'Other Reviewers' section. Webmasters keep a careful eye on traffic leaks and reciprocal links. The review sites themselves do their own self-analysis, because as a critic, even the smallest error becomes hypocritical. Surfers choose which review sites to bookmark and which to avoid. Perhaps when asking who critiques the critics, the quick answer is: "Who doesn't?" It's actually possible that review sites are in fact the most frequently and intensively reviewed entities of the adult affiliate business.

If you are approached by a reviewer there is no substitute for doing your own due diligence and looking at the site where that writer wants to post the review. That initial moment of reviewing the reviewer is essential to determining whether the person you are speaking with is a worthwhile consultant and a quality traffic source, or just another asshole with an opinion.