[MSU Sitemaps] Google Indexing: URLs not followed

Technical difficulties? Ask for help here.
plamoya
Posts: 5
Joined: Fri Feb 08, 2008 7:59 am

[MSU Sitemaps] Google Indexing: URLs not followed

Postby plamoya » Tue Feb 12, 2008 5:56 pm

Problem with zen cart after installing Magic SEO.
So I submitted sitemap.php to google and I expected google to crawl my products in english, japanese and french. but sadly results are all of them are "URLs not followed" status with "Redirect error" for each url. Google provides this help pages : http://www.google.com/support/webmaster ... 5156&hl=en . what should I do to make google crawl my site correctly. This is very critical situation , because we started using Magick SEO for indexing of multilingual content by Google in the first place. Please help!
Last edited by plamoya on Tue Feb 12, 2008 9:24 pm, edited 1 time in total.

plamoya
Posts: 5
Joined: Fri Feb 08, 2008 7:59 am

Re: [MSU Sitemaps] Google Indexing: URLs not followed

Postby plamoya » Tue Feb 12, 2008 6:53 pm

By the way , looking at my apache log access log, my guess is Google access is being redirected twice :

Code: Select all

66.249.73.69 - - [13/Feb/2008:01:48:16 +0900] "GET /fr/hg-gundam-seed-18/gundam-hg-murasame-production-type-190.html HTTP/1.1" 302 26 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.73.69 - - [13/Feb/2008:01:48:16 +0900] "GET /index.php?main_page=product_info&products_id=190 HTTP/1.1" 301 26 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

inveo
Inveo Support
Posts: 1285
Joined: Sat Feb 02, 2008 12:07 pm
Contact:

Re: [MSU Sitemaps] Google Indexing: URLs not followed

Postby inveo » Wed Feb 13, 2008 6:49 am

This problem does not relate with Magic SEO URL, but is caused by incomplete cache data which are used for URL creation in MSU SItemaps. Data for each URL creation are not available until first visit of page occurred and therefore old URLs are displayed for the first time (and HTTP 301 Moved Permanently is used for redirection to nice-urls).
I recommend you to remove Sitemaps from Google Webmaster Tool until these data will be created completely (or you can also try to wait with Sitemaps submitted). It only needs some time.

plamoya
Posts: 5
Joined: Fri Feb 08, 2008 7:59 am

Re: [MSU Sitemaps] Google Indexing: URLs not followed

Postby plamoya » Wed Feb 13, 2008 7:59 am

Strange, I can't find any log entry in access log where Googlebot gets HTTP 200 Status at all! How can I confirm that cache actually works for Googlebot's second access?

inveo
Inveo Support
Posts: 1285
Joined: Sat Feb 02, 2008 12:07 pm
Contact:

Re: [MSU Sitemaps] Google Indexing: URLs not followed

Postby inveo » Wed Feb 13, 2008 11:12 am

You can check it by browsing your site :) Magic SEO URL provides to GoogleBot (or any other bot) nearly same URLs as to user. You can also see all URLs by looking to the Sitemap files listed in Sitemaps Index file (sitemaps.php).

plamoya
Posts: 5
Joined: Fri Feb 08, 2008 7:59 am

Re: [MSU Sitemaps] Google Indexing: URLs not followed

Postby plamoya » Thu Feb 14, 2008 12:05 am

I think it just doesn't work for Googlebot, I made an access to certain page (http://plamoya.com/en/gundam-24/gundam- ... s-392.html)in our site with my Browser ( there should be cache created , I assume), after that I simulated Googlebot access like below and still it responds with Redirect 302 Code!!!
I repeated this for several times and nothing changes!

Code: Select all

telnet plamoya.com 80
GET /en/gundam-24/gundam-20/gundam-00-1-100-scale-gundam-kyrios-392.html HTTP/1.1
Host: plamoya.com
User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

HTTP/1.1 302 Found
Date: Thu, 14 Feb 2008 06:58:40 GMT
Server: Apache/2.2.3 (Unix) mod_ssl/2.2.3 OpenSSL/0.9.8a PHP/5.2.2
X-Powered-By: PHP/5.2.2
Location: http://plamoya.com/index.php?main_page=product_info&products_id=392
Content-Length: 0
Content-Type: text/html; charset=UTF-8

Connection closed by foreign host.

inveo
Inveo Support
Posts: 1285
Joined: Sat Feb 02, 2008 12:07 pm
Contact:

Re: [MSU Sitemaps] Google Indexing: URLs not followed

Postby inveo » Thu Feb 14, 2008 2:24 am

It doesn't look like to be related to MSU, because all redirections are always made with HTTP 301 Moved Permanently - any other HTTP status code including 302 code is overwritten.
You can prove my idea by removing installation line from application_top.php and requesting same nice-url (I am not at my office right now, so I can not test it myself).

inveo
Inveo Support
Posts: 1285
Joined: Sat Feb 02, 2008 12:07 pm
Contact:

Re: [MSU Sitemaps] Google Indexing: URLs not followed

Postby inveo » Thu Feb 14, 2008 7:11 am

I just tested my idea. It really must be caused by some modification(s) or MOD(s) you are using. Please PM or email me your FTP login details. I figure it out ASAP as usually.

sirsonic
Posts: 7
Joined: Wed Jan 21, 2009 4:37 pm

Re: [MSU Sitemaps] Google Indexing: URLs not followed

Postby sirsonic » Wed Jan 21, 2009 4:42 pm

We have been experiencing similar errors lately with Google. http://www.gabrielross.com

Category: Errors for URLs in Sitemaps ‎(2,411)‎ HTTP errors ‎(0)‎ Not found ‎(1)‎ URLs not followed ‎(6,810)‎

All errors are noted as "Redirect Errors". Is there a quick way to do a test to see what Google is complaining about?

inveo
Inveo Support
Posts: 1285
Joined: Sat Feb 02, 2008 12:07 pm
Contact:

Re: [MSU Sitemaps] Google Indexing: URLs not followed

Postby inveo » Wed Jan 21, 2009 4:53 pm

Are you using latest MSU Sitemaps? Download it, wait at least 24h and let us know.