Google Page Depletion Continues
Roy Schestowitz
affirm that Google are not only dropping duplicate pages (e.g.
"www.example.org" and "example.org"), but they are losing actual pages. I
know it's no news, but this trend appears to perpetuate. How much impact
will this have and for how long? Does anyone have any idea why this is going
on? The sites are definitely not offending their cause and some are not even
mine. Is it a fluke? Or an intentional initiative to refine (or
contrariwise, degrade) search results WWW-wide?
tonnie
Hi Roy,
Noticed the same and above that, even noticed a few sites reappear that
had been banned for at least 5 months and should not popup again since
they still use the techniques they were banned for in the first place.
So IMHO Google is kaputt, broken and not to be fixed that easily.
--
Website design:
http://vision2form.nl/websitedesign/
Search engine optimization:
http://vision2form.nl/websitedesign/being-found.html
Borek
>> I have just checked the saturation of 4 Web sites and I can pretty much
>> affirm that Google are not only dropping duplicate pages (e.g.
>> "www.example.org" and "example.org"), but they are losing actual pages.
> Noticed the same and above that, even noticed a few sites reappear that
> had been banned for at least 5 months and should not popup again since
> they still use the techniques they were banned for in the first place.
>
> So IMHO Google is kaputt, broken and not to be fixed that easily.
I am repeating is for at least few weeks now :(
To quote Big Bill:
> I think they out-clever themselves. It's too big, they can't controlit
> either. It's Frankensearch. Matt Cutts comes over like,"It's allcool,
> guys, just a little tweaking" and in reality them PHDs arerunning around
> on the mountain going "Oh my God! It's escaped!" andthe villagers are
> outside with flaming torches and pitchforks...
My (less literary) take is that either they have some nasty bug in the
system, or that their algos got to such complication degree that they have
some emergent behaviours (as in http://en.wikipedia.org/wiki/Emergence)
they can not dampen. Second case is much more interesting, even if it is a
very long shot.
Best,
Borek
--
http://www.chembuddy.com
http://www.ph-meter.info/pH-Nernst-equation
http://www.terapia-kregoslupa.waw.pl
John Bokma
> On Tue, 16 May 2006 10:28:54 +0200, tonnie <t.pr...@chello.nl>
> wrote:
[..]
>> So IMHO Google is kaputt, broken and not to be fixed that easily.
>
> I am repeating is for at least few weeks now :(
Odd... I have the feeling my site is working again, traffic goes up, and
pages appear again at normal speed (or close to) in Google:
<http://www.google.com/search?q=inurl:mexit+inurl:05+inurl:2006
&hl=en&lr=&start=0&sa=N&filter=0>
BTW, I see "Note this" :-) (Google Notebook, which sounds interesting)
--
John Freelance Perl programmer: http://castleamber.com/
Installing and configuring Apache on Windows XP (virtual hosting):
http://johnbokma.com/windows/apache-virtual-hosts-xp.html
Big Bill
<newsg...@schestowitz.com> wrote:
>I have just checked the saturation of 4 Web sites and I can pretty much
>affirm that Google are not only dropping duplicate pages (e.g.
>"www.example.org" and "example.org"), but they are losing actual pages. I
>know it's no news, but this trend appears to perpetuate. How much impact
>will this have and for how long? Does anyone have any idea why this is going
>on?
Just like they screw up everything else when they try to improve it,
Google screwed up Big Daddy. I'm not surprised, everything they do is
necessarily without precedent and we're the test bed. Omelettes and
eggs.
BB
--
http://www.kruse.co.uk/sandbox.htm
http://www.here-be-posters.co.uk/jimi-hendrix-posters.htm
http://www.crystal-liaison.com/armani/index.html
Big Bill
Actually, I mentioned that Frankensearch idea on an email group I'm in
and they were well taken with it. Maybe I should expand it. Ahem...
Google don't do press releases any more, they just say to a serf, "Run
- warn the village!"
You can tell when you're at the Googleplex, the door's opened by a
7-foot high butler called Search. Tee-hee.
John Bokma
> On Tue, 16 May 2006 08:59:36 +0100, Roy Schestowitz
> <newsg...@schestowitz.com> wrote:
>
>>I have just checked the saturation of 4 Web sites and I can pretty
>>much affirm that Google are not only dropping duplicate pages (e.g.
>>"www.example.org" and "example.org"), but they are losing actual
>>pages. I know it's no news, but this trend appears to perpetuate. How
>>much impact will this have and for how long? Does anyone have any idea
>>why this is going on?
>
> Just like they screw up everything else when they try to improve it,
> Google screwed up Big Daddy. I'm not surprised, everything they do is
> necessarily without precedent and we're the test bed. Omelettes and
> eggs.
Google normally does random IP based tests. Also, it's not that hard to
test with 1-2 non-public data centers before going life. And that's what
they did AFAIK. If they really made a huge mistake I am sure they would
have roled back the whole thing.
It's possible that what we see is just a settling down of BD, and that
"over" SEOd site are more affected then others. Like I already wrote, I
have the feeling Google is catching up. The # of visitors/day on my site
was almost stable for 2+ months, and new pages were added very slowly
(days and days).
And now, for 2-3 weeks I see them go up and up again, 11,680 Monday, and I
expect 12,000+ next week :-D. Not bad, since last year it was 3,800+/day
max for this month.
--
John
Net::Google and Perl: http://johnbokma.com/perl/net-google.html
Roy Schestowitz
> Big Bill <kr...@cityscape.co.uk> wrote:
>
>> On Tue, 16 May 2006 08:59:36 +0100, Roy Schestowitz
>> <newsg...@schestowitz.com> wrote:
>>
>>>I have just checked the saturation of 4 Web sites and I can pretty
>>>much affirm that Google are not only dropping duplicate pages (e.g.
>>>"www.example.org" and "example.org"), but they are losing actual
>>>pages. I know it's no news, but this trend appears to perpetuate. How
>>>much impact will this have and for how long? Does anyone have any idea
>>>why this is going on?
>>
>> Just like they screw up everything else when they try to improve it,
>> Google screwed up Big Daddy. I'm not surprised, everything they do is
>> necessarily without precedent and we're the test bed. Omelettes and
>> eggs.
Relax, Bill, relax. Google do the best they can and they remain the best bar
none.
I'm having my lunch as I write this. No omelettes, no eggs though/
> Google normally does random IP based tests. Also, it's not that hard to
> test with 1-2 non-public data centers before going life. And that's what
> they did AFAIK. If they really made a huge mistake I am sure they would
> have roled back the whole thing.
>
> It's possible that what we see is just a settling down of BD, and that
> "over" SEOd site are more affected then others. Like I already wrote, I
> have the feeling Google is catching up. The # of visitors/day on my site
> was almost stable for 2+ months, and new pages were added very slowly
> (days and days).
>
> And now, for 2-3 weeks I see them go up and up again, 11,680 Monday, and I
> expect 12,000+ next week :-D. Not bad, since last year it was 3,800+/day
> max for this month.
I wondered if this entire wave of depletion was affecting only large sites,
but I doubt it because I checked some smaller sites whose number of indexed
page I occasionally track. No wonder you're doing so well, John... with key
phrases like 'calapa sex' and 'mexican sperm'.
Best wishes,
Roy
--
Roy S. Schestowitz, Ph.D. Candidate (Medical Biophysics)
http://Schestowitz.com | GNU/Linux Ś PGP-Key: 0x74572E8E
1:10pm up 18 days 20:07, 8 users, load average: 0.03, 0.05, 0.00
http://iuron.com - next generation of search paradigms
Roy Schestowitz
> On Tue, 16 May 2006 10:58:13 +0200, Borek
> <m.bor...@delete.chembuddy.these.com.parts> wrote:
>
>>On Tue, 16 May 2006 10:28:54 +0200, tonnie <t.pr...@chello.nl> wrote:
>>
>>>> I have just checked the saturation of 4 Web sites and I can pretty much
>>>> affirm that Google are not only dropping duplicate pages (e.g.
>>>> "www.example.org" and "example.org"), but they are losing actual pages.
>>
>>> Noticed the same and above that, even noticed a few sites reappear that
>>> had been banned for at least 5 months and should not popup again since
>>> they still use the techniques they were banned for in the first place.
>>>
>>> So IMHO Google is kaputt, broken and not to be fixed that easily.
Could anyone comment on patterns of crawling and volume of crawling, from
Google in particular? The number of indexed pages falls by the day and there
is little flux at the moment. The loss propagates to all datacentres, not
just Big Daddy. It makes this seem irreversible (not trivially reversible so
anyway).
>>I am repeating is for at least few weeks now :(
>>
>>To quote Big Bill:
>>
>>> I think they out-clever themselves. It's too big, they can't controlit
>>> either. It's Frankensearch. Matt Cutts comes over like,"It's allcool,
>>> guys, just a little tweaking" and in reality them PHDs arerunning around
>>> on the mountain going "Oh my God! It's escaped!" andthe villagers are
>>> outside with flaming torches and pitchforks...
*LOL* Darn, I missed that one...
>>My (less literary) take is that either they have some nasty bug in the
>>system, or that their algos got to such complication degree that they have
>>some emergent behaviours (as in http://en.wikipedia.org/wiki/Emergence)
>>they can not dampen. Second case is much more interesting, even if it is a
>>very long shot.
>>
>>Best,
>>Borek
>
> Actually, I mentioned that Frankensearch idea on an email group I'm in
> and they were well taken with it. Maybe I should expand it. Ahem...
> Google don't do press releases any more, they just say to a serf, "Run
> - warn the village!"
> You can tell when you're at the Googleplex, the door's opened by a
> 7-foot high butler called Search. Tee-hee.
*LOL*
--
Roy S. Schestowitz
http://Schestowitz.com | Free as in Free Beer Ś PGP-Key: 0x74572E8E
1:15pm up 18 days 20:12, 8 users, load average: 0.05, 0.12, 0.05
http://iuron.com - semantic engine to gather information
Big Bill
<newsg...@schestowitz.com> wrote:
>__/ [ John Bokma ] on Tuesday 16 May 2006 12:26 \__
>
>> Big Bill <kr...@cityscape.co.uk> wrote:
>>
>>> On Tue, 16 May 2006 08:59:36 +0100, Roy Schestowitz
>>> <newsg...@schestowitz.com> wrote:
>>>
>>>>I have just checked the saturation of 4 Web sites and I can pretty
>>>>much affirm that Google are not only dropping duplicate pages (e.g.
>>>>"www.example.org" and "example.org"), but they are losing actual
>>>>pages. I know it's no news, but this trend appears to perpetuate. How
>>>>much impact will this have and for how long? Does anyone have any idea
>>>>why this is going on?
>>>
>>> Just like they screw up everything else when they try to improve it,
>>> Google screwed up Big Daddy. I'm not surprised, everything they do is
>>> necessarily without precedent and we're the test bed. Omelettes and
>>> eggs.
>
>
>Relax, Bill, relax.
I am relaxed Goddammit!!
>I wondered if this entire wave of depletion was affecting only large sites,
>but I doubt it because I checked some smaller sites whose number of indexed
>page I occasionally track. No wonder you're doing so well, John... with key
>phrases like 'calapa sex' and 'mexican sperm'.
I was going to have my lunch but, suddenly, somehow, my appetite seems
to have deserted me...:-)) Mexican sperm, eh? Maybe John's little guys
have handlebar mustaches and will leer a lot at your sisters.
I can feel a top ten list coming on here...
John Bokma
> I was going to have my lunch but, suddenly, somehow, my appetite seems
> to have deserted me...:-)) Mexican sperm, eh? Maybe John's little guys
> have handlebar mustaches and will leer a lot at your sisters.
> I can feel a top ten list coming on here...
:-) Well, I get also hits for "horse xxx"
--
John Experienced (web) developer: http://castleamber.com/
Firefox RSS: http://johnbokma.com/firefox/rss-and-live-bookmarks.html
Big Bill
Oh, I imagine I'll be saying it again...
Big Bill
>Big Bill <kr...@cityscape.co.uk> wrote:
>
>> I was going to have my lunch but, suddenly, somehow, my appetite seems
>> to have deserted me...:-)) Mexican sperm, eh? Maybe John's little guys
>> have handlebar mustaches and will leer a lot at your sisters.
>> I can feel a top ten list coming on here...
>
>:-) Well, I get also hits for "horse xxx"
Probably only in reference to the one you rode in on...
Paul B
wrote:
>On Tue, 16 May 2006 13:21:31 +0100, Roy Schestowitz
><newsg...@schestowitz.com> wrote:
>
>>__/ [ Big Bill ] on Tuesday 16 May 2006 11:52 \__
>>
>>> On Tue, 16 May 2006 10:58:13 +0200, Borek
>>> <m.bor...@delete.chembuddy.these.com.parts> wrote:
>>>
>>>>On Tue, 16 May 2006 10:28:54 +0200, tonnie <t.pr...@chello.nl> wrote:
>>>>
>>>>>> I have just checked the saturation of 4 Web sites and I can pretty much
>>>>>> affirm that Google are not only dropping duplicate pages (e.g.
>>>>>> "www.example.org" and "example.org"), but they are losing actual pages.
<snip>
>>
>>*LOL* Darn, I missed that one...
>
>Oh, I imagine I'll be saying it again...
>
>BB
We know :/
--
----== Posted via Newsfeeds.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
----= East and West-Coast Server Farms - Total Privacy via Encryption =----
John Bokma
> On 16 May 2006 12:54:25 GMT, John Bokma <jo...@castleamber.com> wrote:
>
>>Big Bill <kr...@cityscape.co.uk> wrote:
>>
>>> I was going to have my lunch but, suddenly, somehow, my appetite seems
>>> to have deserted me...:-)) Mexican sperm, eh? Maybe John's little guys
>>> have handlebar mustaches and will leer a lot at your sisters.
>>> I can feel a top ten list coming on here...
>>
>>:-) Well, I get also hits for "horse xxx"
>
> Probably only in reference to the one you rode in on...
Nah, I have a picture of a horse on my site, and some young people post
stuff like:
OMG!!!!!11111 ur horse is lovely!!!!1111111
xxx
Stacey
:-D (IIRC, that one had a German IP address, or something like that).
--
John Perl programmer: http://johnbokma.com/perl/perlprogrammer.html
TextPad+TortoiseSVN:http://johnbokma.com/textpad/textpad-subversion.html
Phil Payne
I thought it was someone using Inspector Clouseau's pronunciation of
"berm".
John Bokma
Not really, a male Mexican tarantula made a sperm web :-) But why I have a
hit for Mexican sperm is beyond me.
--
John Experienced (web) developer: http://castleamber.com/
Textpad quick reference card (pdf): http://johnbokma.com/textpad/
Roy Schestowitz
> "Phil Payne" <ph...@isham-research.co.uk> wrote:
>
>>> Mexican sperm, eh?
>>
>> I thought it was someone using Inspector Clouseau's pronunciation of
>> "berm".
>
> Not really, a male Mexican tarantula made a sperm web :-) But why I have a
> hit for Mexican sperm is beyond me.
Could be anchor text. Have you got some female fans? That "omg! ur horse is
cute" girl might have to become a prime suspect.
--
Roy S. Schestowitz
http://Schestowitz.com | Free as in Free Beer Ś PGP-Key: 0x74572E8E
4:30pm up 18 days 23:27, 8 users, load average: 1.02, 0.56, 0.42
tonnie
> "Phil Payne" <ph...@isham-research.co.uk> wrote:
>
>
>>>Mexican sperm, eh?
>>
>>I thought it was someone using Inspector Clouseau's pronunciation of
>>"berm".
>
>
> Not really, a male Mexican tarantula made a sperm web :-) But why I have a
> hit for Mexican sperm is beyond me.
Well John, let me explain;
- sperm and mexican in the title
- sperm and mexican in the text
- sperm in several alt attributes
- sperm in the URl
need i say more... :)
Borek
<newsg...@schestowitz.com> wrote:
> Could anyone comment on patterns of crawling and volume of crawling, from
> Google in particular? The number of indexed pages falls by the day and
> there
> is little flux at the moment. The loss propagates to all datacentres, not
> just Big Daddy. It makes this seem irreversible (not trivially
> reversible so anyway).
AFAIK all datacenters are BigDaddy now.
Two different sites (month, numbers of visits):
01 3450
02 1490
03 1908
04 1590
05 466
so May will be below 1000. Note that this site doesn't have dropped pages
problem and new pages added at the end of April (or was it beginning of
May?) were spidered and indexed in about 10 days.
01 270
02 241
03 356
04 88
05 55
May will be comparable to April, perhaps slightly better. Note that this
site has only half pages indexed and despite sitemap listing all pages
Googlebot have not spidered these non-indexed pages since March IIRC. They
don't differ from any other page on the site.
Roy Schestowitz
> On Tue, 16 May 2006 14:21:31 +0200, Roy Schestowitz
> <newsg...@schestowitz.com> wrote:
>
>> Could anyone comment on patterns of crawling and volume of crawling, from
>> Google in particular? The number of indexed pages falls by the day and
>> there
>> is little flux at the moment. The loss propagates to all datacentres, not
>> just Big Daddy. It makes this seem irreversible (not trivially
>> reversible so anyway).
>
> AFAIK all datacenters are BigDaddy now.
>
> Two different sites (month, numbers of visits):
>
> 01 3450
> 02 1490
> 03 1908
> 04 1590
> 05 466
Wow. I guess I am among the luckier ones. I didn't lose much traffic, but I
find the number rather unnerving. They keep sliding and referrals likewise.
Oddly, I had a huge boost for a couple of weeks last month.
One of the minor losses (to me) is the inability to carry out site search, on
which I depend. I put a lot of my papers and meterial on-line. This
frequently, if not immediately, enables me to 'offshore' indexing to Google
(rather than Beagle, Spotlight or Google Desktop locally) and then navigate
through my work. At the moment, some pages are simply missing. I must
confess that Google's 'sort by relevance' was spectacular, so I could arrive
at my desired location instantly, without fumbling.
> so May will be below 1000. Note that this site doesn't have dropped pages
> problem and new pages added at the end of April (or was it beginning of
> May?) were spidered and indexed in about 10 days.
>
> 01 270
> 02 241
> 03 356
> 04 88
> 05 55
>
> May will be comparable to April, perhaps slightly better. Note that this
> site has only half pages indexed and despite sitemap listing all pages
> Googlebot have not spidered these non-indexed pages since March IIRC. They
> don't differ from any other page on the site.
I have never truly considered sitemaps because the sites are fairly
fragmented, CMS-wise. I am relieved to discover that even XML could not save
the day, so to speak. Many of us feel your pain...
Best wishes,
Roy
--
Roy S. Schestowitz
http://Schestowitz.com | Open Prospects Ś PGP-Key: 0x74572E8E
5:40pm up 19 days 0:37, 8 users, load average: 0.94, 0.89, 0.63
http://iuron.com - knowledge engine, not a search engine
canadafred
<snip>
> Could anyone comment on patterns of crawling and volume of crawling, from
> Google in particular? The number of indexed pages falls by the day and there
> is little flux at the moment. The loss propagates to all datacentres, not
> just Big Daddy. It makes this seem irreversible (not trivially reversible so
> anyway).
<snip>
I monitor about 14 sites daily. About two months ago I noticed that
Google crawled the web sites much less frequently, from every second
day to every second week. This happened at about the same time as I
started bi-weekly Google SiteMaps refreshes. So, I figured the change
in crawling frequency was affected by how often I refreshed the Google
SiteMaps on the servers. Google seemed to pick up every page, every
image, everything I gave it.
Suddenly, an ancient web site that I had been trying to resurrect over
the past year sprang back into life. Keyphrases began appearing in top
results; their improvent and presence sustained.
Where are all those powerful mega-paged competitors gone? I thought
that surely they had only stepped out of their throned positions for
but a brief moment. No. They seem to have been evicted permanently,
much to my chagrin.
--
Fred
http://www.fly-in-fishing.net/
pushing the limits of fishing in Canada
Darren Tipton
Depletion Continues":
>Could anyone comment on patterns of crawling and volume of crawling, from
>Google in particular? The number of indexed pages falls by the day and there
>is little flux at the moment. The loss propagates to all datacentres, not
>just Big Daddy. It makes this seem irreversible (not trivially reversible so
>anyway).
Can only comment on my advanced driving site.
Hasn't been Google crawled since March.
Every data center now shows between 67 and 70 pages in the index. Was over
900.
SERP varies greatly from datacenter to datacenter for the same search term.
On some data center's I'm at pos 7, on some pos 10, on other's 73. It also
changes from check to check of the same data center !
Same number of backlinks on every data center.
Total site traffic is up 16% on last month....approx 21,000 page requests
this week.
Darren
--
Darren Tipton - Remove the fruit to reply by mail
Google Datacenter Checker: http://www.tippy.co.uk/google-datacenter-check/
Advanced Google PR Compare: http://www.tippy.co.uk/page-rank-compare/
How to Be an Advanced Driver: http://www.advanced-driving.co.uk/bb/
Borek
<newsg...@schestowitz.com> wrote:
>>> Could anyone comment on patterns of crawling and volume of crawling,
>>> from
>>> Google in particular? The number of indexed pages falls by the day and
>>> there
>>> is little flux at the moment. The loss propagates to all datacentres,
>>> not
>>> just Big Daddy. It makes this seem irreversible (not trivially
>>> reversible so anyway).
>>
>> AFAIK all datacenters are BigDaddy now.
>>
>> Two different sites (month, numbers of visits):
>>
>> 01 3450
>> 02 1490
>> 03 1908
>> 04 1590
>> 05 466
>
>
> Wow. I guess I am among the luckier ones. I didn't lose much traffic,
> but I
> find the number rather unnerving. They keep sliding and referrals
> likewise.
> Oddly, I had a huge boost for a couple of weeks last month.
That was crawling, not visitors. Visitors number seems to be more or less
constant. Second site is sandboxed yet, so the statistics are not too
convincing - but Google visitors number is lower for sure.
Big Bill
<ph...@isham-research.co.uk> wrote:
They go bermp in the night.
Big Bill
wrote:
>John Bokma wrote:
>> "Phil Payne" <ph...@isham-research.co.uk> wrote:
>>
>>
>>>>Mexican sperm, eh?
>>>
>>>I thought it was someone using Inspector Clouseau's pronunciation of
>>>"berm".
>>
>>
>> Not really, a male Mexican tarantula made a sperm web :-) But why I have a
>> hit for Mexican sperm is beyond me.
>
>Well John, let me explain;
>
>- sperm and mexican in the title
>- sperm and mexican in the text
>- sperm in several alt attributes
>- sperm in the URl
>
>need i say more... :)
Sperm with everything.... after three, lads ...sperm, sperm, sperm,
sperm...
Big Bill
wrote:
>On Tue, 16 May 2006 14:01:32 GMT, Big Bill <kr...@cityscape.co.uk>
>wrote:
>
>>On Tue, 16 May 2006 13:21:31 +0100, Roy Schestowitz
>><newsg...@schestowitz.com> wrote:
>>
>>>__/ [ Big Bill ] on Tuesday 16 May 2006 11:52 \__
>>>
>>>> On Tue, 16 May 2006 10:58:13 +0200, Borek
>>>> <m.bor...@delete.chembuddy.these.com.parts> wrote:
>>>>
>>>>>On Tue, 16 May 2006 10:28:54 +0200, tonnie <t.pr...@chello.nl> wrote:
>>>>>
>>>>>>> I have just checked the saturation of 4 Web sites and I can pretty much
>>>>>>> affirm that Google are not only dropping duplicate pages (e.g.
>>>>>>> "www.example.org" and "example.org"), but they are losing actual pages.
>
><snip>
>>>
>>>*LOL* Darn, I missed that one...
>>
>>Oh, I imagine I'll be saying it again...
>>
>>BB
>
>We know :/
and again...:-))))
Big Bill
<newsg...@apple.tippy.org> wrote:
>On Tue, 16 May 2006 13:21:31 +0100, Roy Schestowitz
><newsg...@schestowitz.com> wrote on the topic of "Re: Google Page
>Depletion Continues":
>
>>Could anyone comment on patterns of crawling and volume of crawling, from
>>Google in particular? The number of indexed pages falls by the day and there
>>is little flux at the moment. The loss propagates to all datacentres, not
>>just Big Daddy. It makes this seem irreversible (not trivially reversible so
>>anyway).
>
>Can only comment on my advanced driving site.
>
>Hasn't been Google crawled since March.
>
>Every data center now shows between 67 and 70 pages in the index. Was over
>900.
>
>SERP varies greatly from datacenter to datacenter for the same search term.
>On some data center's I'm at pos 7, on some pos 10, on other's 73. It also
>changes from check to check of the same data center !
>
>Same number of backlinks on every data center.
>
>Total site traffic is up 16% on last month....approx 21,000 page requests
>this week.
>
>Darren
So, you're now better off than you used to be, right? Maybe this is a
genuine attempt by Google to encourage us away from the concept of
gaming their algo to get to the top of the serps and instead to sit
back and observe best practice and let them by use of some obscure and
as yet understood method bring us more visitors than we formerly had
when our sites were listed as indexed and we had high serps.
I'm serious (again!), this could be their way of dealing with spam,
taking away the incentive to be at the top of the search results. Why
did we all want to be there in the first place? Because that way
supposedly we got the most visitors. Now maybe they are presenting us
with an algo that mysteriously gets us more visitors than if we were
at the top so there's no point spamming to get there any more.
John Bokma
> __/ [ John Bokma ] on Tuesday 16 May 2006 16:23 \__
>
>> "Phil Payne" <ph...@isham-research.co.uk> wrote:
>>
>>>> Mexican sperm, eh?
>>>
>>> I thought it was someone using Inspector Clouseau's pronunciation of
>>> "berm".
>>
>> Not really, a male Mexican tarantula made a sperm web :-) But why I
>> have a hit for Mexican sperm is beyond me.
>
> Could be anchor text. Have you got some female fans? That "omg! ur
> horse is cute" girl might have to become a prime suspect.
:-s
--
John Experienced (web) developer: http://castleamber.com/
Perl RSS Builder: http://johnbokma.com/perl/rss-web-feed-builder.html
John Bokma
> John Bokma wrote:
>> "Phil Payne" <ph...@isham-research.co.uk> wrote:
>>
>>
>>>>Mexican sperm, eh?
>>>
>>>I thought it was someone using Inspector Clouseau's pronunciation of
>>>"berm".
>>
>>
>> Not really, a male Mexican tarantula made a sperm web :-) But why I
>> have a hit for Mexican sperm is beyond me.
>
> Well John, let me explain;
>
> - sperm and mexican in the title
> - sperm and mexican in the text
> - sperm in several alt attributes
> - sperm in the URl
>
> need i say more... :)
That I understand, but what had that visitor in mind when looking for
Mexican sperm (and do I really want to know).
--
John Experienced (web) developer: http://castleamber.com/
Perl RSS Builder: http://johnbokma.com/perl/rss-web-feed-builder.html
www.1-script.com
> Could anyone comment on patterns of crawling and volume of crawling,
> from Google in particular?
Leisurely (lazy?) crawling explains best what I see on all but one site.
On this one site Googlebot behaves like Slurp, gobbling up thousands of
pages per day and actually already going circles 'cause I think there are
no more new pages left. There is no reason I can identify for that strange
behavior except maybe a software bug? Did I forget to mention that the
crawled pages never make it into the index?
> The number of indexed pages falls by the day and
> there is little flux at the moment. The loss propagates to all datacentres,
> not just Big Daddy.
I think Matt Cutts was little too quick to call Big Daddy "complete". They
therefore lost a good opportunity to blame all problems on the deployment
issues. So, officially Big Daddy == Google. Has been since late March.
> It makes this seem irreversible (not trivially
> reversible so
> anyway).
It sure does seem that way here, too. It will soon be one month after I've
cleaned some redirect and canonical issues on my largest site and no
improvements have been observed yet.
--
Cheers,
Dmitri
See Site Sig Below
--
+------------------------------------------------+
| Follow alt.internet.search-engines threads |
| with your Firefox Live Bookmarks! Set it up at |
| http://www.1-script.com/forums/ |
+------------------------------------------------+
www.1-script.com
> So, you're now better off than you used to be, right? Maybe this is a
> genuine attempt by Google to encourage us away from the concept of
> gaming their algo to get to the top of the serps and instead to sit
> back and observe best practice and let them by use of some obscure and
> as yet understood method bring us more visitors than we formerly had
> when our sites were listed as indexed and we had high serps.
How's that possible, I wonder? You've got knocked down to the third page
that only 5% of users ever visit yet you still get same traffic? If you
are not on top of the SERPs, you are not getting any traffic, period. So
the name of the game is still the same.
> I'm serious (again!), this could be their way of dealing with spam,
> taking away the incentive to be at the top of the search results. Why
> did we all want to be there in the first place? Because that way
> supposedly we got the most visitors. Now maybe they are presenting us
> with an algo that mysteriously gets us more visitors than if we were
> at the top so there's no point spamming to get there any more.
You are seriously kidding, right? Randomizing results as a way to combat
spam? SERP #1 page is supposed to be the most relevant. If 5 seconds later
it gets "randomized" down to SERP #21, did it become less relevant?
You are probably giving them too much credit. If something is happening,
it does not necessarily mean that it's under their control.
I personally think that there could couple possible reasons for the
current problems:
#1 they managed to loose a huge chunk of their database on a large enough
number of datacenters couple months ago. Now they are struggling to sync
the datacenters having to make do with a smaller subset of the database in
the meantime.
#2 "Big Daddy"-style indexer's performance is considerably less than that
of the old one's. Since the start of the Big Daddy in Nov '05()? crawlers
bring more data than indexers can actually parse/analyze/store due to more
sophisticated analysis every page has to go through. Web grows
exponentially yet I believe they can only buy new servers at a particular
rate because AdWords income should have flatten out by now. So, given no
technological breakthroughs lately, they are not going to catch up. Or at
the very least it is not going to be easy.
www.1-script.com
> So, you're now better off than you used to be, right? Maybe this is a
> genuine attempt by Google to encourage us away from the concept of
> gaming their algo to get to the top of the serps and instead to sit
> back and observe best practice and let them by use of some obscure and
> as yet understood method bring us more visitors than we formerly had
> when our sites were listed as indexed and we had high serps.
How's that possible, I wonder? You've got knocked down to the third page
that only 5% of users ever visit yet you still get same traffic? If you
are not on top of the SERPs, you are not getting any traffic, period. So
the name of the game is still the same.
> I'm serious (again!), this could be their way of dealing with spam,
> taking away the incentive to be at the top of the search results. Why
> did we all want to be there in the first place? Because that way
> supposedly we got the most visitors. Now maybe they are presenting us
> with an algo that mysteriously gets us more visitors than if we were
> at the top so there's no point spamming to get there any more.
You are seriously kidding, right? Randomizing results as a way to combat
www.1-script.com
> Could anyone comment on patterns of crawling and volume of crawling,
> from Google in particular?
Leisurely (lazy?) crawling explains best what I see on all but one site.
On this one site Googlebot behaves like Slurp, gobbling up thousands of
pages per day and actually already going circles 'cause I think there are
no more new pages left. There is no reason I can identify for that strange
behavior except maybe a software bug? Did I forget to mention that the
crawled pages never make it into the index?
> The number of indexed pages falls by the day and
> there is little flux at the moment. The loss propagates to all datacentres,
> not just Big Daddy.
I think Matt Cutts was little too quick to call Big Daddy "complete". They
therefore lost a good opportunity to blame all problems on the deployment
issues. So, officially Big Daddy == Google. Has been since late March.
> It makes this seem irreversible (not trivially
> reversible so
> anyway).
It sure does seem that way here, too. It will soon be one month after I've
cleaned some redirect and canonical issues on my largest site and no
improvements have been observed yet.
--
Roy Schestowitz
> Big Bill wrote:
>
>> So, you're now better off than you used to be, right? Maybe this is a
>> genuine attempt by Google to encourage us away from the concept of
>> gaming their algo to get to the top of the serps and instead to sit
>> back and observe best practice and let them by use of some obscure and
>> as yet understood method bring us more visitors than we formerly had
>> when our sites were listed as indexed and we had high serps.
>
> How's that possible, I wonder? You've got knocked down to the third page
> that only 5% of users ever visit yet you still get same traffic? If you
> are not on top of the SERPs, you are not getting any traffic, period. So
> the name of the game is still the same.
Agreed. It is around 5% for page 3 and provided that sites lose their status
(as perceived by the algorithms), they will inevitably fail to top the
SERP's. The number of indexed pages has become rather ridiculous. While I
still get decent traffic from Google (Honest! I am yet to investigate the
reason**), 99% of my pages appear not to reside in the index (site:URL), but
when I search for phrases, the pages are there. It seems like some anomaly
or inconsistency in the algorithms (essentially a mistake). This isn't the
case with every site that I check, so others were lucky. Some sites that I
look at have hundreds of thousands of pages. All still there...
**Maybe it's the "I suffer, but so does my enemy" point of equilibrium.
Before I started 'losing' pages, I had a fortnight of sudden # of referrals
surge.
>> I'm serious (again!), this could be their way of dealing with spam,
>> taking away the incentive to be at the top of the search results. Why
>> did we all want to be there in the first place? Because that way
>> supposedly we got the most visitors. Now maybe they are presenting us
>> with an algo that mysteriously gets us more visitors than if we were
>> at the top so there's no point spamming to get there any more.
>
> You are seriously kidding, right? Randomizing results as a way to combat
> spam? SERP #1 page is supposed to be the most relevant. If 5 seconds later
> it gets "randomized" down to SERP #21, did it become less relevant?
*LOL*
> You are probably giving them too much credit. If something is happening,
> it does not necessarily mean that it's under their control.
Especially when you talk about tera- and perabyte datacentres, which work
like horses and chew bandwidth 24/7.
> I personally think that there could couple possible reasons for the
> current problems:
>
> #1 they managed to loose a huge chunk of their database on a large enough
> number of datacenters couple months ago. Now they are struggling to sync
> the datacenters having to make do with a smaller subset of the database in
> the meantime.
That'd be an 'ouch'. Could they have pulled out some old stuff from backup?
> #2 "Big Daddy"-style indexer's performance is considerably less than that
> of the old one's. Since the start of the Big Daddy in Nov '05()? crawlers
> bring more data than indexers can actually parse/analyze/store due to more
> sophisticated analysis every page has to go through. Web grows
> exponentially yet I believe they can only buy new servers at a particular
> rate because AdWords income should have flatten out by now. So, given no
> technological breakthroughs lately, they are not going to catch up. Or at
> the very least it is not going to be easy.
Maybe the Register was right after all. Maybe they are choking on all that
spam -- that which used to be excluded. I have always estimated that only
about 10% of the Web was indexed. This could explain the crawling lull that
Borek, yourself and I are witnessing.
Best wishes,
Roy
--
Roy S. Schestowitz | Windows all-in-one: Word, IE (for E-mail) & iTunes
http://Schestowitz.com | GNU is Not UNIX Ś PGP-Key: 0x74572E8E
7:35am up 19 days 14:32, 8 users, load average: 0.73, 0.49, 0.43
http://iuron.com - proposing a non-profit search engine
Big Bill
>tonnie <t.pr...@chello.nl> wrote:
>
>> John Bokma wrote:
>>> "Phil Payne" <ph...@isham-research.co.uk> wrote:
>>>
>>>
>>>>>Mexican sperm, eh?
>>>>
>>>>I thought it was someone using Inspector Clouseau's pronunciation of
>>>>"berm".
>>>
>>>
>>> Not really, a male Mexican tarantula made a sperm web :-) But why I
>>> have a hit for Mexican sperm is beyond me.
>>
>> Well John, let me explain;
>>
>> - sperm and mexican in the title
>> - sperm and mexican in the text
>> - sperm in several alt attributes
>> - sperm in the URl
>>
>> need i say more... :)
>
>That I understand, but what had that visitor in mind when looking for
>Mexican sperm (and do I really want to know).
Possibly a donation!
BB (Eeeeew!)
Big Bill
>Roy Schestowitz wrote:
>
>> Could anyone comment on patterns of crawling and volume of crawling,
>> from Google in particular?
>
>Leisurely (lazy?) crawling explains best what I see on all but one site.
>On this one site Googlebot behaves like Slurp, gobbling up thousands of
>pages per day and actually already going circles 'cause I think there are
>no more new pages left. There is no reason I can identify for that strange
>behavior except maybe a software bug? Did I forget to mention that the
>crawled pages never make it into the index?
>
>> The number of indexed pages falls by the day and
>> there is little flux at the moment. The loss propagates to all datacentres,
>> not just Big Daddy.
>
>I think Matt Cutts was little too quick to call Big Daddy "complete".
Frankensearch! Ha!
> They
>therefore lost a good opportunity to blame all problems on the deployment
>issues. So, officially Big Daddy == Google. Has been since late March.
>
>> It makes this seem irreversible (not trivially
>> reversible so
>> anyway).
>
>It sure does seem that way here, too. It will soon be one month after I've
>cleaned some redirect and canonical issues on my largest site and no
>improvements have been observed yet.
It's too big, I tells ya! It's all got away from them now.
BB
John Bokma
> On 17 May 2006 01:14:26 GMT, John Bokma <jo...@castleamber.com> wrote:
[ .. ]
>>That I understand, but what had that visitor in mind when looking for
>>Mexican sperm (and do I really want to know).
>
> Possibly a donation!
LOL!
--
John Freelance Perl programmer: http://castleamber.com/
Firefox Keywords: http://johnbokma.com/firefox/keymarks-explained.html
Paul B
>Big Bill <kr...@cityscape.co.uk> wrote:
>
>> On 17 May 2006 01:14:26 GMT, John Bokma <jo...@castleamber.com> wrote:
>
>[ .. ]
>
>>>That I understand, but what had that visitor in mind when looking for
>>>Mexican sperm (and do I really want to know).
>>
>> Possibly a donation!
>
>LOL!
Well, it's better than making a withdrawal ;)
Big Bill
wrote:
>On 17 May 2006 17:09:29 GMT, John Bokma <jo...@castleamber.com> wrote:
>
>>Big Bill <kr...@cityscape.co.uk> wrote:
>>
>>> On 17 May 2006 01:14:26 GMT, John Bokma <jo...@castleamber.com> wrote:
>>
>>[ .. ]
>>
>>>>That I understand, but what had that visitor in mind when looking for
>>>>Mexican sperm (and do I really want to know).
>>>
>>> Possibly a donation!
>>
>>LOL!
>
>Well, it's better than making a withdrawal ;)
We'll be taking your word for that then!
BB
Paul B
wrote:
>On Wed, 17 May 2006 12:19:26 -0500, Paul B <lamewo...@yahoo.com>
>wrote:
>
>>On 17 May 2006 17:09:29 GMT, John Bokma <jo...@castleamber.com> wrote:
>>
>>>Big Bill <kr...@cityscape.co.uk> wrote:
>>>
>>>> On 17 May 2006 01:14:26 GMT, John Bokma <jo...@castleamber.com> wrote:
>>>
>>>[ .. ]
>>>
>>>>>That I understand, but what had that visitor in mind when looking for
>>>>>Mexican sperm (and do I really want to know).
>>>>
>>>> Possibly a donation!
>>>
>>>LOL!
>>
>>Well, it's better than making a withdrawal ;)
>
>We'll be taking your word for that then!
>BB
Why ? do you want to take away Mexican sperm ? :o
John Bokma
> On Wed, 17 May 2006 18:47:35 GMT, Big Bill <kr...@cityscape.co.uk>
> wrote:
>
>>On Wed, 17 May 2006 12:19:26 -0500, Paul B <lamewo...@yahoo.com>
>>wrote:
>>
>>>On 17 May 2006 17:09:29 GMT, John Bokma <jo...@castleamber.com> wrote:
>>>
>>>>Big Bill <kr...@cityscape.co.uk> wrote:
>>>>
>>>>> On 17 May 2006 01:14:26 GMT, John Bokma <jo...@castleamber.com>
>>>>> wrote:
>>>>
>>>>[ .. ]
>>>>
>>>>>>That I understand, but what had that visitor in mind when looking
>>>>>>for Mexican sperm (and do I really want to know).
>>>>>
>>>>> Possibly a donation!
>>>>
>>>>LOL!
>>>
>>>Well, it's better than making a withdrawal ;)
>>
>>We'll be taking your word for that then!
>>BB
>
> Why ? do you want to take away Mexican sperm ? :o
So it was Bill then? :-D
--
John Freelance Perl programmer: http://castleamber.com/
A better start menu with Quick Launch:
http://johnbokma.com/windows/quick-launch.html