Crawling, indexing, and ranking Google Group

Re: Wordpress, Zenphoto and a tonne of 404s

2008-09-07T02:23:17Z

Looks like I'm in a mess! I don't even know where to start!

In the case of...

[link]...
[link]...

the actual image on the server is...

Flirting Between Lost & Found v1.jpg

Re: Wordpress, Zenphoto and a tonne of 404s

2008-09-07T02:10:56Z

I believe there is a MAC equivalent to Xenu, though maybe not as good.

Somebody posted about it recently ... hmmmm... need to find that post.

Ok, there is this:
[link]

And here:
[link]

Re: Wordpress, Zenphoto and a tonne of 404s

2008-09-07T02:04:11Z

Webado,

I replied before I read your last post. I'll go through it now and
post again in a bit.

Re: Wordpress, Zenphoto and a tonne of 404s

2008-09-07T02:02:29Z

Webado,

Thanks for the invaluable information!

I checked out Xenu. Is there a Mac equivalent?

Also, I changed my robots.txt to what you recommended. I also have one
for the Zenphoto part of the site, in which I have...

User-agent: *
Allow: /gallery/albums
Disallow: /gallery/cache
Disallow: /gallery/themes

Re: Reconsideration request - have we done enough ?

2008-09-07T01:54:37Z

Thanks BBDeath - we'll be looking to tighten things up a lot more with
all of your good inputs.

Re: Wordpress, Zenphoto and a tonne of 404s

2008-09-07T01:51:40Z

OK, from the gallery folder and below you have 21 broken links (for
images), when crawling with Xenu:
[link]

Broken links, ordered by link:
[link]
error code: 404 (not found), linked from page(s):

Re: Wordpress, Zenphoto and a tonne of 404s

2008-09-07T01:29:48Z

Keep in mind that you will get 404's for any brokne links apeparing
elsewhere on the web. As long as your site has no such broken links
NOW, don't worry about them, they will drop out in time. Or try to
capture and 301 redirect them to the correct url.
It looks like those malformed links got cached maybe from a tempoarary

Re: Wordpress, Zenphoto and a tonne of 404s

2008-09-07T01:26:19Z

With a Wordpress blog you need to block certain urls in robots.txt.
For instance anythign from the folders suffixed by wp- .
So the robots.txt fiel shoudl be at least:

User-agent: *
Disallow: /wp-

Re: Wordpress, Zenphoto and a tonne of 404s

2008-09-07T01:21:53Z

Did you scan the site using Xenu?

Also your robots.txt is not quite right.

You have:

User-Agent: *
Allow: /
Allow: /gallery/albums
Allow: /albums/
Allow: /gallery/

Sitemap: [link]

It doesn't seem you intend to block anything, so this should get
rewritten as:

Re: Wordpress, Zenphoto and a tonne of 404s

2008-09-07T01:06:02Z

Thanks for looking! Some of the urls may no longer exist - however
there shouldn't be more than a dozen or so of those. I've been working
on the problem for some time and have changed things around to try to
identify the issue.

As it stands now Webmaster says I have 735 errors for urls and 743 not

Re: Multipule Questions

2008-09-07T00:45:11Z

It's indexed:
[link]

Whether you can find the site by any search terns you are trying or
not is a different story. That depends on how well optimised the site
is for whatever searches you want to rank for, how many backlinks you

Re: Problem with Text Only Cache

2008-09-07T00:41:49Z

Thanks so much! I will try and validate it.

Re: What could still be causing my penalty?

2008-09-07T00:36:46Z

"go to a real forum like webmasterworld"

Now that is funny.

Re: Googlebot killing one of my servers

2008-09-07T00:33:16Z

If this account /~hlstatsx/ does not exist it should issue a different
response, not a 404 simply for the url not existing.
Maybe a 403 or a 500.

Multipule Questions

2008-09-07T00:31:22Z

First I verified my site about a week ago. Today I was asked to verify
again. Do I have to do this every time I log in? If not how frequently
do I have to do it?

Second google says that they indexed my site ( [link]
) on the 1st. they also say im included in the google index. However