Google Sitemaps

Official Google Webmaster Central Blog

Official news on crawling and indexing sites for the Google index


What's new with Sitemaps.org?

What has the Sitemaps team been up to since we announced sitemaps.org? We've been busy trying to get Sitemaps adopted by everyone and to make the submission process as easy and automated as possible. To that end, we have three new announcements to share with you.

First, we're making the sitemaps.org site available in 18 languages! We know that our users are located all around the world and we want to make it easy for you to learn about Sitemaps, no matter what language you speak. Here is a link to the Sitemap protocol in Japanese and the FAQ in German.

Second, it's now easier for you to tell us where your Sitemaps live. We wondered if we could make it so easy that you wouldn't even have to tell us and every other search engine that supports Sitemaps. But how? Well, every website can have a robots.txt file in a standard location, so we decided to let you tell us about your Sitemap in the robots.txt file. All you have to do is add a line like

Sitemap: http://www.mysite.com/sitemap.xml

to your robots.txt file. Just make sure you include the full URL, including the http://. That's it. Of course, we still think it's useful to submit your Sitemap through Webmaster tools so you can make sure that the Sitemap was processed without any issues and you can get additional statistics about your site

Last but not least, Ask.com is now also supporting the Sitemap protocol. And with the ability to discover your Sitemaps from your robots.txt file, Ask.com and any other search engine that supports this change to robots.txt will be able to find your Sitemap file.

  |  Links to this post   |  Post a Comment  |   




Links to this post:

Create a Link

Comments:
Great idea! I don't speak Japanese, but the robots.txt and Ask news are very useful. My only question was whether Google would still report stats in webmaster central for autodiscovery, but the post seems rather clear that we should still submit manually to get those. Thanks again!
 
Vanessa, do the united engines support multiple sitemap locations in robots.txt? Lots of folks maintaining a handful of sitemaps don't mess with sitemap indexes.
Thanks!
 
That's all very good.

I still wonder why Blogger can't auto-generate a REAL Sitemap.xml file, rather than have us resort to hacking around with incomplete Feed URLs.

It seems so very obvious and trivial for Blogger to produce such a file, when it knows about all our posts on the blog.
 
Awesome! Can we still use a .txt sitemap file using the robots.txt protocol for sitemaps? I ask becase the example is .xml and I see no mention of text files.
 
Gareth:
All content already saved at Blogger, why save them again?
 
Why can't we just add a < link> tag?

E.g.:

< link rel="index" href="sitemap.xml" title="Sitemap" charset="UTF-8" type="application/xml" />

That's what the < link> tag is meant to do, and the type="application/xml" can be used to tell it's a sitemap in the Sitemaps.org format.

(Note: I wrote "< link" (with a space) because Blogger would not allow the posting of a tag)
 
Specifying the location of the sitemap within the robots.txt file gives the following parsing result -Syntax not understood.

Is this just temporary?

Thanks.
 
protocol page in traditional chinese is not fully translated!

from Informing search engine crawlers to end of page.
 
We run a couple of different domains off a single webroot which share the same robots.txt file. Is it okay to have multiple Sitemap: entries in a robots.txt file? Or what would you recommend
 
This is a great thing! This will make life a lot easier for folks like me who run huge sites with thousands of pages.
 
Since reading an artilce about the importance of sitemaps, I have created sitemaps for each of my web sites. I think it's great that Google provides us with the information we need to create these important sitemaps. Thanks Google! Thank you Vanessa for keeping us informed.
 
"Specifying the location of the sitemap within the robots.txt file gives the following parsing result -Syntax not understood. Is this just temporary?"

I have the same question.
 
Vanessa, it's a simple and a great idea, especially because of the standard use of robot.txt, every webmaster need a unique way to submit a sitemap to multiple engines.
I tried to include this directive:
Sitemap: http://www.mysite.com
to my websites but "Webmaster Tools" under section "robot.txt analysis" now said that my robot.txt has a bad syntax.
Webmaster tool still does not accept sitemap?
 
Good to see more use being made of Robots.txt. How about incorporating other useful hints for search engines such as a geographic location to ensure sites are listed in the correct country search. (This could not be abused as there's only one robots.txt file per site and only one location would be accepted.)

I'm in the UK, but my sites are hosted in the US because it's cheaper.
 
This is awesome! I've been using the Google Webmaster Tools for quite a while now and I love it. However, I still can figure out why Google only has 6 pages indexed from my site (http://www.menscience.com), when I have over 600 pages of really good content. Can someone please explain this to me and give some suggestions. Thanks!
 
I like that sitemaps.org is now available in Dutch, however, I noticed a wrong translation at http://www.sitemaps.org/nl/faq.html#faq_sitemap_location

Hope someone can correct that.
 
Just wondering if what is the reason to force webmasters to provide FULL paths (starting with http:// ) to sitemap files?! Why the "Sitemap:" entry format should NOT comply with "Disallow:" entry format that requires RELATIVE paths?! You can't have a sitemap file coming from one server for another server anyway; so what is the point?
Also, why there is no way to specify multiple sitemaps in one row like multiple "Disallow" paths separated by space?!
 
Someone at www.sitemaps.org renamed the /0.9/ directory into /09/ so all the feeds can't be validated against http://www.sitemaps.org/schemas/sitemap/0.9/ no more. Can someone fix it?
 
Good !dea, but other features wich are available on google sitemaps are completely missed out like forbidden pages, crawl date etc. while we get clear idea about other things using google sitemaps.

Thanks
 
Can we write also sitemap.gz?
 
thanks for sharing !
 
Post a Comment

Previous Posts: