Home > Features > Serenading The Bots • Bookmark   • Newsletters   • Register Search Options


Serenading The Bots

Serenading The Bots

April 13, 2005
Text size: 
Get XBIZ News
XBIZ Research
Will virtual reality boost the paysite market?
Yes, it will soon
Yes, but in a few years
Out of 150 votes. Results based on votes submitted by members of XBIZ.net social network.

" Google will spider galleries, so will MSN, but Yahoo is reluctant "

Everybody has their own way of dealing with search engine bots, and while I'm not saying that the following is God's own truth, these techniques might be effective, if used properly. Having said that, let's get on with my newest tips for SEO'ing your website: Order and recognizing the behavioral patterns of each individual SE spider.

We all know search engines love text, but if there's one thing which comes close to text as an SE spider favorite, it is order. Bots love alphabetically arranged links, date-wise arranged links and category-wise arranged links, in that order.

Let's say you have a ton of links – to galleries, other link sites, hosted galleries, etc. Start with a list of your categories, arranged alphabetically, which we'll use for the index page. Now, for link sites, free hosted galleries and TGPs, arrange your links first by date, and within each date, by alphabetical order. So that's Category -> Date -> Alphabet. For your own in-house galleries, skip the date. This works wonders in ways you won't be able to imagine – I started getting hits for keywords I didn't even know were on my site!

Here's an example: You have a list of galleries: Dirty Aly, Dirty Daisy, Dirty Fuckers, Dirty Juliet. Now the title of your page says "Sex Galleries." So on a search for "'dirty,' 'something,' 'sex' and/or 'galleries'," you have a better chance of making the results, because whatever you have in your page title counts, the number of times you have 'dirty' in one sentence counts and whatever you have in the URL counts. You can't change the URL, and it's not sound practice to change the page title, but you can add value to your search results by keeping your links alphabetical.

SE Spider Individuality
Each bot - GoogleBot, Yahoo, Slurp, MSNbot and all the others, have things they do, and things they won't do; and it varies from site to site. The trick is to find out the limitations of your site, vis-à-vis each bot, and then act accordingly. Here's a few practical examples:

Bot Visit Frequency: How often does a particular bot visit your site and/or pages? If most of your pages are spidered every day, then you don't have to think too much about it. But if only your index page is spidered everyday, and the other pages maybe once a month or so, then it needs a little tweaking. Fill up your index page with keyword-rich text to attract hits to the index page.

Make the links which you want spidered at the top of your page 'bold' and link to the important pages from every other page you can. This way, when the spider wants to go through the rest of your pages, it knows which page is important.

PHP: The GoogleBot won't go through PHP, unless your content is unique and cannot be found elsewhere and it considers the page to be very important. The Yahoo and MSN bots have no such sensitivities, and will eat up PHP the same as HTML. If you have a PHP page which generates another HTML page, the Yahoo bot will follow the HTML page to its source PHP page and spider that one too. The MSNbot will even follow sites starting from dynamic trade URLs.

Galleries: For most adult websites, galleries make up a big part of the total content, so it makes a big difference if your galleries are spidered or not. Find out which bot is visiting your galleries, and cater to that bot. Keep it happy by adding new galleries, text, categories, order, whatever. If your galleries start showing up in search results, it will give a big boost to your search engine traffic.

Google will spider galleries, so will MSN, but Yahoo is reluctant unless the gallery is being linked to from a lot of external places, particularly on other domains. This is why most 'free adult sites' with in-house galleries receive more traffic from Google than Yahoo, while for websites with text-based and other content, the traffic from both Google and Yahoo might be closer to each other.

Revisit After Meta Tag: A small matter, but one that has enormous impact on the behavior of a bot, is that if you add a "Revisit After 'X' Days" meta tag, you need to update that page every 'X' days. Once the bot finds the tag, and comes back to the page after 'X' days, and then finds no update, it will not visit the page again until it's indexing your whole site, so any further updates you make to that page are pretty useless.

If you are not sure whether a page is going to updated, or if the page has static content which is unlikely to change, then it's better not to put in the "Revisit After" meta tag.

A systematic layout and a good index page gives results not only in search engines, but also for an enhanced surfer experience with increased bookmarkers and free link-backs from other sites and webmasters who enjoy your site and would like to share it with their surfers. Not to mention the fact that it makes it easy for you to find something on your own site...


WIA Profile: Mia Saldarriaga

Each month, industry news media organization XBIZ spotlights the career accomplishments and outstanding contributions of Women in Adult. WIA profiles offer an intimate look at the professional lives of... More »

Profile: CAM4 Pushes Boundaries in Adult

CAM4 remains a leading live-streaming adult show platform in what has quickly become a very competitive field. Although CAM4 launched as a brand in 2007, the company behind it has been involved in the... More »

ExoClick’s Publishing Head Is Passionate About Job

ExoClick’s Ada LLlorca, who leads publishing efforts for the ad network, sure loves the adult webmaster space. And maybe that’s why she can honestly say she’s good friends with the company’s... More »
Stay informed of the latest industry developments. Get XBIZ newsletters delivered to your inbox. Subscribe today!
Enter email address:

* To manage existing subscriptions click here.

Submit your press release to
multiple news outlets with 1 click.
Subscribe to RSS news feeds or
add free content to your website.
Access XBIZ news and articles
with your mobile device.
Subscribe to XBIZ Premiere magazine, the industry's leading adult retail trade publications, delivering the most timely and comprehensive business news and information to producers and retailers of adult products.



Nov 04 - Nov 06
Edison, New Jersey

2016 Inked Awards

Nov 05 - Nov 05
Edison, New Jersey

XBIZ 2017

Jan 09 - Jan 13
Hollywood, CA

XBIZ Awards 2017

Jan 12 - Jan 12
Los Angeles, CA
Everyday thousands of business professionals browse XBIZ's industry directory for quality products and services. Not listed yet? Your company could be losing potential new business. Submit your company today!
Use XBIZ RSS feeds to stay informed of the latest industry developments or as a content syndication tool for your website!