Google SiteMaps - Letting the search engines know you mean business!

There's been some really big news breaking on the SEO front in the past couple of weeks. Yahoo and Microsoft have announced that they're going to come to the party with the Google SiteMaps protocol. For those of you who thought you'd never get all those search engine giants together in the same room, here they are!

So what is a sitemap and why should you want one?

At a very basic level, a sitemap is just a list of the pages on your website. You're probably familiar with the for-humans type of sitemaps you find on complex, complicated or difficult to navigate websites. It's typically a bunch of lists pointing to every single page in your website.

Sitemaps are also good for search engines for the same reasons. They allow search engine spiders to crawl your site faster and more thoroughly, and to find pages that might not be easy to get to, or (for one reason or another) might not be getting indexed easily otherwise. At a very basic level, your typical for-humans sitemap will also help the search engine spiders without your intervention, but more savvy webmasters are submitting their sitemaps to the search engines. In particular, they're using Google's Sitemaps to give Google even more information about the nature of the various pages on their site so that Google can crawl them more effectively.

What is Google Sitemaps?

Google Sitemaps is a tool that allows webmasters to give the search engines information about their sites -- not just URLs for pages contained on the site, but also information about how frequently the page is updated, and the relative importance of the page compared to other pages on the website. The information is contained in an XML file that resides on your web server, and you can submit the sitemap to Google so it knows where to find it.

So, spell it out for me?

* Creating a sitemap according to Google's sitemap protocol allows you to specify how frequently a particular page on your site is updated. It might be hourly, daily, weekly, monthly or yearly. This gives a little hint to Google as to how frequently the page should be crawled. Of course, *hint* is the key word here. Google doesn't *obey* sitemaps, it merely takes your recommendations into consideration.

* You can also specify the relative importance of a page on your site, compared to other pages on your site. This is most useful when Google is returning multiple results from the same domain. By telling Google which pages are most important, you can be sure that they won't display your privacy policy (or other uninteresting information!) when it should be displaying your sales page.

* Submitting your sitemap not only tells Google where to find this extremely useful document, but gives Google a heads up as to your site's existence if it's new and doesn't have many links to it yet. It's also useful for keeping the search engines abreast of new additions to your website, if you're updating it regularly (which you should be!).

Here's what Google has to say about sitemaps.

And what's the excitement?

The excitement is that Yahoo and Microsoft are adopting Google's sitemap protocol, so the information previously only available to Google will now be available to all the major search engines. You'll be able to create one sitemap, submit it to all three search engines, and soon have them all crawling every important page of your site the best way possible. That's good news.

How to I get a sitemap?

Google provides a tool you can run on your server to generate a sitemap, but it seems quite complicated. There are reasonably comprehensive instructions on writing the XML file yourself on Sitemaps.org. If you're not too keen on writing it yourself, there are free tools available on the internet for generating XML sitemaps. Simply searching for "sitemap generator" pulls up a number of sites. One thing to note with these generators is that they will typically enter general responses into the "how frequently is this page updated?" and "what's the importance of this page?" sections. So, for instance, if you've got a reasonably static site that is updated monthly, but ALSO a blog that gets updated daily, you'll want to manually edit the XML file to change the value from "monthly" to "daily" for the blog URL. Similarly, if you've got pages that are more or less important than others, you'll need to edit that data yourself.

That all sounds pretty dramatic if you're not particularly tech-savvy, but really once your XML file is generated, it's just a matter of going through it and looking for the URLs of pages you're particularly interested in. Soon after the URL you'll find the information you need to modify. The good thing about XML (as opposed to other programming languages) is that it's mostly in English, with a few angled brackets here and there. It's pretty easy to read, and you can just edit it with a text-editor like notepad or BBEdit. (You need to save the file with UTF-8/Unicode encoding. Both of these programs will allow you to do that, if you select the option when you're saving. Don't use MS Word to do this :) )  

For more information about the Google Sitemap protocol, Google, Yahoo and Microsoft have teamed up to produce the (pretty dry) Sitemaps.org . To submit your sitemap to Google you should visit their Webmaster Tools section.

Good luck everyone!

Mark