Creepy Crawlies: Part One


Spiders and Crawlers are your best friend! So be as nice to them as possible! Really! It's in your own long-term best interest. If you can't get yourself listed on Search Engines, you're missing out on a lot of extra $$$.

The thing is, it's easy to make your site Search Engine friendly and exponentially increase your exposure. Within 1 month of starting a brand-new domain, I was receiving hits from one of the most rapidly growing Search Engines — Google. I've heard rumors that AltaVista hasn't actually updated it's Database since July 2001. That's a long time. That's also why Google is taking over the market.

And guess what — Google updates it's Index every month on the 22nd. That's why I made sure I had the Crawler Friendly parts of my site ready prior to the 22nd. Even though the site hadn't been completed, Google indexed a whole lot of information about it and by the 1st of the month, visitors were rolling in. So, I want to share a few tricks with you on how to make the most of this and other Search Engines and how to optimize your site to get noticed by them.

I'm still amazed how many big pay sites have little or no Meta Data. Using it effectively can increase your profits by 100% at least. All you need to do is add a few tags to your Splash page and preferably any other publicly viewable page:

title>XXX Competitions - Win XXX stuff every day!/title

Make your Title tag as descriptive as you want (without going overboard). Search Results will display 50 or so characters, but they will index more than that — so say what you need to say in the first sentence and then add another sentence or two.

meta name="description" content="XXX Competitions - Win XXX stuff every day! We have all XXX Competitions on the Internet in one place. Add your own competition today!"

Similar rules apply for the Description tag. Keep the opening concise and then write a few Keyword saturated sentences.

meta name="keywords" content="porn, free, XXX, competitions, win:."

Don't overdo your Keywords as many Engines now penalize you for Keyword Stuffing. Stick to around 20-30 good Keywords. Try experimenting with different combinations on different pages.

meta name="Robots" content="index,follow"
meta name="MSSmartTagsPreventParsing" content="TRUE"
meta name="revisit-after" content="10 days"
meta name="ROBOTS" content="ALL"

These tags basically say to the Crawler "Index Me Please!"

meta name="last-modified" content="06-02-02"
meta name="author" content="admin@"
meta name="host" content=""
meta name="host-admin"content=""
meta name="copyright" content="© 2002 All rights reserved. Material herein is the property::"

These tags are fairly self explanatory and although they're not supported by all Crawlers yet, they are becoming increasingly popular in Ranking Algorithms. Every little bit of extra exposure helps!

Unfortunately, most Pay Sites, while providing heaps of content for their visitors, don't provide content for Search Engines. Crawlers need Keywords, blocks of text and alternate descriptions on images to build their database. Your content is generally locked away behind a .htaccess file and so the Crawlers never see it. Your front page and tour are generally graphic based and don't give the Crawlers much to index.

A few simple ways of making yourself more visible are —

Make sure you add an ALT tag to every image — be as descriptive as you want. It helps with your Keyword Saturation and also re-enforces your site's branding/message to the visitor while the site is loading i.e. the little text descriptions that are displayed while the images load. If your sponsor is ok with it, ALT tag their banners using the text on the banner — it means they're going to make more money too because you're more visible to Crawlers.

On your front page, right down at the bottom, make a link to a file called something like site-map.html. On this page you should create a straight HTML page that lists every main Index File in your site and also a link to every other site you run. The Crawler will follow the link to your Site Map and then crawl everything it finds on there. This page should link to all your TGP's, AVS sites and basically EVERYTHING you have that isn't hidden behind a password protected area.

Make a text based version of your front pages and add a link to them on your Site Map. It may not convert as well as the graphic one, but it will add heaps of Keywords to the Search Engine databases. If you're clever — make a hybrid mixture of the two. Just make sure you include that link on your Site Map.

Do this to every site you own!!! Search Engines LOVE sites that are linked to by other sites. Link Popularity is increasingly used to determine rankings so the more cross-links you have, the more they'll like you. Provided you've set up your 404 traps well, you can make a good bit of extra revenue from this method.

Consider adding some extra Free Content to the non-protected areas of your site. You don't need to link to them from anywhere that your visitors can see (you don't want them wasting precious bandwidth), but make sure these content pages link back to your main page and that they are linked to from your Site Map and vice-versa. Either add an Exit Console that leads to your main site and/or brand it heavily enough to ensure they click through to the main site.

Timing Is Everything
Remember how I said that Google crawls on the 22nd of each month? Well, I want to ensure that I get as much content indexed as possible when it crawls each time.

I recently implemented a Site Of The Day program (SOTD) — I recommend one of my Sponsors site's to my visitors every day — a nice simple revenue raiser. Here's the sneaky thing that I did though; I'm building the pages for all my SOTD through to the end of NEXT month prior to THIS month's Google crawling and then linking them all through the Site Map.

Although the SOTD will not be shown to my visitors for another 30 days (or more if I can get 2 months worth of pages made), they are indexed in Google NOW and people are finding them + finding my main site as a result.

Do I care if a surfer looks at the SOTD file destined for next month? Heck no, he may click on it and join the site or he might go to my main site. Either way, I've increased my earning potential NOW while planning for the future. And I've lost nothing.

In a way, you also have a psychological edge on your visitor. He feels like he's stumbled onto something he shouldn't have and may continue to search for things in the hope of finding more illicit goodies. Provided you've set up your 404 traps well, you can make a good bit of extra revenue from this method.

There are many other ways of increasing your Search Engine exposure that only require a little effort to integrate into your current site — you just have to be creative and be prepared to sacrifice a little bandwidth in return for much higher placement in Search Engine rankings. The results are well worth it though — especially if you're in this for the long-haul.

In my next article I'll give you some further hints on increasing your Search Engine exposure.