[MSU Sitemaps] Google Indexing: URLs not followed

[plamoya]

Problem with zen cart after installing Magic SEO.
So I submitted sitemap.php to google and I expected google to crawl my products in english, japanese and french. but sadly results are all of them are "URLs not followed" status with "Redirect error" for each url. Google provides this help pages : http://www.google.com/support/webmaster ... 5156&hl=en . what should I do to make google crawl my site correctly. This is very critical situation , because we started using Magick SEO for indexing of multilingual content by Google in the first place. Please help!

[plamoya]

By the way , looking at my apache log access log, my guess is Google access is being redirected twice :

Code: Select all

66.249.73.69 - - [13/Feb/2008:01:48:16 +0900] "GET /fr/hg-gundam-seed-18/gundam-hg-murasame-production-type-190.html HTTP/1.1" 302 26 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.73.69 - - [13/Feb/2008:01:48:16 +0900] "GET /index.php?main_page=product_info&products_id=190 HTTP/1.1" 301 26 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

[inveo]

This problem does not relate with Magic SEO URL, but is caused by incomplete cache data which are used for URL creation in MSU SItemaps. Data for each URL creation are not available until first visit of page occurred and therefore old URLs are displayed for the first time (and HTTP 301 Moved Permanently is used for redirection to nice-urls).
I recommend you to remove Sitemaps from Google Webmaster Tool until these data will be created completely (or you can also try to wait with Sitemaps submitted). It only needs some time.

[plamoya]

Strange, I can't find any log entry in access log where Googlebot gets HTTP 200 Status at all! How can I confirm that cache actually works for Googlebot's second access?

[inveo]

You can check it by browsing your site :) Magic SEO URL provides to GoogleBot (or any other bot) nearly same URLs as to user. You can also see all URLs by looking to the Sitemap files listed in Sitemaps Index file (sitemaps.php).

[plamoya]

I think it just doesn't work for Googlebot, I made an access to certain page (http://plamoya.com/en/gundam-24/gundam- ... s-392.html)in our site with my Browser ( there should be cache created , I assume), after that I simulated Googlebot access like below and still it responds with Redirect 302 Code!!!
I repeated this for several times and nothing changes!

Code: Select all

telnet plamoya.com 80
GET /en/gundam-24/gundam-20/gundam-00-1-100-scale-gundam-kyrios-392.html HTTP/1.1
Host: plamoya.com
User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

HTTP/1.1 302 Found
Date: Thu, 14 Feb 2008 06:58:40 GMT
Server: Apache/2.2.3 (Unix) mod_ssl/2.2.3 OpenSSL/0.9.8a PHP/5.2.2
X-Powered-By: PHP/5.2.2
Location: http://plamoya.com/index.php?main_page=product_info&products_id=392
Content-Length: 0
Content-Type: text/html; charset=UTF-8

Connection closed by foreign host.

[inveo]

It doesn't look like to be related to MSU, because all redirections are always made with HTTP 301 Moved Permanently - any other HTTP status code including 302 code is overwritten.
You can prove my idea by removing installation line from application_top.php and requesting same nice-url (I am not at my office right now, so I can not test it myself).

[inveo]

I just tested my idea. It really must be caused by some modification(s) or MOD(s) you are using. Please PM or email me your FTP login details. I figure it out ASAP as usually.

[sirsonic]

We have been experiencing similar errors lately with Google. http://www.gabrielross.com

Category: Errors for URLs in Sitemaps ‎(2,411)‎ HTTP errors ‎(0)‎ Not found ‎(1)‎ URLs not followed ‎(6,810)‎

All errors are noted as "Redirect Errors". Is there a quick way to do a test to see what Google is complaining about?

[inveo]

Are you using latest MSU Sitemaps? Download it, wait at least 24h and let us know.