[Zen Cart] Errors with Sitemaps

[rginn]

Hi,

Just got magic a few days ago and you guys installed it form (Thanks).
I have a question about the google webmaster tools. I tried posting over there but no one cares. Anyway, I have on the sitemap page a red "X". It states there are 107 URLS submitted, and 2 have been indexed. But when I hover over the red "X" it says Errors, but I dont have a clue what the errors are. I check day after day but nothing else gets indexed. Can you help?
I am using IBP tools now and what a nice set of tools. My site is working fine. I have checked every URL and every image pop up and I cant find any problems at least from a user perspective. Any help would be much appreciated.

Thanks
Bob Ginn
LCS

[inveo]

Please post exact error message.

[rginn]

Hi, I just figured out how to look at the errors on Webmaster tools.
The error stated there were areas of the sitemap that were restricted by my robots.txt file. I now have removed my robots file just to test. After looking at the schema file, it seems since it appears dynamic, I need to approach the restrictions in a different way, but not sure how. Do you need to use the htaccess file for each location? If so, where can I go to learn how to do this? I am just guessing here so can you help?

Thanks
Bob

[inveo]

Please post your robots.txt file content.

[rginn]

Well, I deleted it and was going to start over.
As far as pages, I wanted all categories, products, EZpages, About Us, contact us etc.
The only ones I wanted to disallow were new products, featured products, login etc. I guess just the ones that would not offer any positive content. Is there anyway you could post a zencart example robots.txt and then I could go from there?

Thanks
Bob

[inveo]

If you are using MSU, your robots.txt file can be basically blank.

You can start with following content:

Code: Select all

User-agent: *
Disallow: /images/
Disallow: /includes/
Disallow: /download/

Sitemap: http://www.yourdomain.com/sitemaps.php

[rginn]

Hi, thanks again for your time. I do not know much about the file structure of zencart, that is why I am having difficulty. As an example, I never thought I would disallow /includes, as this seems to be where much of the useable file structure is, at least that is one folder that I go to for modifications I need.

As an example I go here for meta_tags:
/public_html/includes/languages/english/override

and here for my main page changes:
/public_html/includes/languages/english/html_includes/override

I guess I am confused why I would disallow /includes.
When looking at sitemap:

sitemaps.php?showMap=Allproducts -- I need this
sitemaps.php?showMap=Categories -- I need this
sitemaps.php?showMap=Ezpages -- I need this
sitemaps.php?showMap=Featured-- This is not turned on in DB Admin
sitemaps.php?showMap=Index -- Not sure what this is
sitemaps.php?showMap=Newproducts -- This is not turned on in DB Admin
sitemaps.php?showMap=Products-- I need this
sitemaps.php?showMap=Reviews-- This is not turned on in DB Admin
sitemaps.php?showMap=Specials-- This is not turned on in DB Admin

I do want the About Us, Contact Us, Privacy, Copyright (legal verbeage) and others which
contain valid indexable content.

I dont want coupons, login, and any other stuff that would not have any pertinent content.
I know that you provide support for your product, and I do feel like I am asking for more than what I should be asking for, but if you could point me to someone that does completely understand the robots.txt file and Magic, that may be able to help me, I would be very grateful.
If the search engine can go everywhere, ot may choose to index something I dont want indexed and miss something I do.

Regards
Bob

[inveo]

It is hard to explain for non-technical users why robots.txt looks this way, but it is exactly what you need.

[rginn]

Thank you!!

Ok I will use the one you posted. I would like to be able to refine it , so hopefully I can find an example. I just worry that google and others may not index the whole site and leave out those pages that I really need indexed.

Thanks Again!
Bob