As a last hope for my business, I've described my situation in here today - hoping that someone from Google would see my mistake and maybe guide me - in my particular situation.
I've written a lot of postings, because I'm truly desperately trying to save my business and the jobs of 4 people that rely on this netshop. I've also provided some information which should help prove that I'm a good girl, never wanting to have any trouble with Google.
As it turns out, when I just visited my webmaster tools, it had happened. My site is getting excluded again - and this time without any e-mail to hint what the problem is. The webmaster tools just says that "no pages from your site are indexed". Sure enough, my competitors with the same site structure (even more popular than mine) is still in business. This almost feels like harrasment. To write a re-inclusion request, I would have liked to know what problem I have, so I can be sure to get rid of it.
I can barely believe this and even though it could just be a coincidence, it looks more like what was predicted to me earlier today in private e-mail. This can't be what I get for trying to bring my site into compliance with Google and saving my business. Google please tell me it isn't so. Google.. I cry for help.. Don't be Microsoft!
Hi Sussie That message is shown to almost everyone at the moment. I have it in all the sites that I checked (and they are still fully indexed). Don't let that message get you down, check for yourself. Checking at http://oy-oy.eu/google/pages/?url=http%3a%2f%2fcasanovafurniture.dk it shows 6 pages indexed (not that a number like that would get your spirits up). It also shows "about 30-40" which tends to hint at duplicates or pages which are otherwise filtered (or just a glitch).
oh, thanks for this information. Earlier I saw this message only during the exclusion period. Sorry if I freaked out.
I even wrote a re-inclusion request now, so I hope it won't be considered spamming.
Has anyone considered that these web-spammers might be given too much attention, at the cost of the serious webmasters. Maybe the balance is tipping. At least I don't feel too good.
Yes, it's pointing me to the guidelines as well. There is at least one other thread about it here in the groups, probably some more elsewhere.
>Has anyone considered that these web-spammers might be given too much >attention, at the cost of the serious webmasters.
Might be. But at the risk of being insensitive (sorry) - I think this group would be filled to the top with people complaining if sites left and right were removed from the index automatically. Each legitimate site is tragedy and I don't think they do it lightly (but who knows).
>From :"Using Rank Propagation and Probabilistic Counting for Link-Based
>Fortunately, web spam detection can be more strict than >e-mail spam detection. While losing a relevant e-mail message >is very bad, demoting a relevant Web page in the search results >is not so bad, because if the page is relevant, it can be found later >by following links, or it can be moved to the second or third >page of results.
Another similar train of thought is: if you rank high (or have a high PR) then you need a lot of good links. If those links do not bring you traffic, are they really good? Would those links have been better placed elsewhere (where you do get traffic)?
You need to get your site into good technical order so that you can achieve the position your site deserves in Google SERPs.
With respect, I suggest that if you want to rescue your site, you need to focus on the problems with your site and not "waste" valuable time speculating about "spammers" and how Google does this or that.
Your site is very faulty - need to fix it as soon as possible so that your business is protected.
Here are some suggestions based on a very quick visit to your site:
1. You should immediately fix the errors in the headsection of all your pages. The person who codes your pages should use validator.w3.org You it would be very helpful if DOCTYPE and character encoding are declared on every page. For example, look at http://www.casanovafurniture.dk/shop/default.asp The page cannot be validated. It defaults to UTF-8, and then fails because it contains "illegal characters" (non-UTF-8).
Solution: Either declare a DOCTYPE and UTF-8 for example:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> and make sure all the rest of the page uses only UTF-8.
** OR ** declare a DOCTYPE and another encoding, for example:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
IF YOU ARE UNSURE WHICH TO USE, use the second one above or something similar that will allow for your Danish characters.
2. Look at your existing site in Firefox and other browsers, not only Internet Explorer. In IE it looks OK, but in Firefox (which is much more standards-compliant), you will see lots of "?" symbols instead of the Danish special-character symbols - you can fix this problem easily.
3. It would be far better if you eventually moved away from framesets. They create a lot of extra problems and complications which your webmaster has clearly not dealt with.
4. Another example of the technical problems on your site: Request this URL (part of a frameset): http://www.casanovafurniture.dk/shop/productinfo.asp?id=15%2D000001x&... Notice in the URL it says ?id=.... Google Guidelines explicitly state that you must not use ID= in URL querystrings (I do not know why). Looking at the source code for the above page, I see: a. invalid head section - you must have a properly formed head section. b. hundreds of lines of javascript *before* the head section c. very little content for a massive amount of coding. Of course this page (and many more) fails w3 validation - but can easily be fixed but it takes time and skill.
I hope this gives you enough evidence that your site has significant technical problems that only you and your webteam can fix - it is not Google's problem or Google's shortcomings.
I am emphatically *not* an apologist for Google, but you and your website-coding-staff need to take "ownership" of this one.
I'm sorry if this sounds unkind - at least you have a response with specific evidence and specific suggestions. No amount of sympathy will solve the problems that are screaming loudly!
It's okay with me that you sound unkind, if only there would also be some substance in your words.
Unfortunately it's clear you haven't read my full story, so your comments are way off.
I've tried to avoid speculation in here, cause everyone has an opinion about what the problem is - and I too can speculate.
What I'm asking of Google is, how would they suggest that a site like mine is published and then I can go back to my shop provider and try to influence them.
I'll give you a few hints, to show you where you are wrong:
- First, this is Denmarks largest shop system and not something I have coders working on. (I've explained elsewhere that I'm bound to this provider and have very limited options. Also, it's better to fix the problems than to just flea them).
- I'm coming here as a last resort. I have already tried different approaches, but they have all failed. The option I then selected as a last resort, is the one that works for my competitor. For example I've earlier used the noframes tag solution, but it got indexed wrong and I lost all 1400 products from Google that way.
- All I do, is fill into products into a form-based shop front end, and then I can tweak some options, just like the hundreds of other shops that works fine there. I listed the options I have, in another article in here.
- My front page is NOT totally empty. It's a frameset.
- My site is easily crawlable. Try with the tools that are out there and see for yourself. Follow the bottom-left link on the front page, and you'll see.
- I've addressed the issue with frames. I cannot move away from them quickly, because I don't control this. It was carefully explained why, in another posting. (By February, I'm forced out of business, so only Google can really help me now).
- The DOCTYPE is not the problem. Study Matt's videos for yourself.
- Your information about the head section and the amounts of code in relation to content is your own speculation. Study Matt Cutts videos, where he addresses this as well (I know, because I've seen them all 3 times or so:). Matt Cutts specifically said that W3 validation isn't important and not worth worrying about. Google is used to pages of varying quality and I'm sure that's not the reason why my shop got excluded. Wouldn't you agree?
- What you say about ID= I would like to know more about. Is it a fact? Where have you seen that? How can it work with thousands of shops in the world? (I think I heard mention of google might thinking it's a session ID, but I don't know how that affects things). Anyway, the easiest would be for Google to tell me this, so I don't lose my whole career over some detail. I'm not alone with this problem. 5 shops were excluded. The e-mail from google said a front page 301 redirect was the cause, but that is not making sense to anyone.
- I'm not a beginner and I'm after the facts now. Not speculation.
Sussie wrote: > Unfortunately ... your comments are way off. > I'll give you a few hints, to show you where you are wrong: > I'm coming here as a last resort. I have already tried different > approaches, but they have all failed. > - My front page is NOT totally empty. It's a frameset.
Sussie, what matters to Google is what they found when the Googlebot visited your site.
When I click to see the "cache" view of these pages (this is what Google "saw" when it visited), I see that 3 pages are blank, 1 is an error page (an error message from your server, not Google, not w3) and one is not cached by Google. These pages caches were based on visits on FOUR different dates!
(On a very minor point, I also notice that 4 of these 6 pages have the same head title.)
> - My site is easily crawlable. Try with the tools that are out there > and see for yourself. Follow the bottom-left link on the front page, > and you'll see.
The link to the onscreen-sitemap? Well, if I access that from www.casanovafurniture.dk it is visible but it contains many HUNDREDS of links (Google limit 100) and many HUNDREDS of images (severe bandwidth degradation).
If I access the onscreen-sitemap from www.casanovafurniture.dk/shop/default.asp, the sitemap is not properly visible in Internet explorer or Firefox (frames in the way - could it be, I wonder, surprise, surprise, a coding error?)
> - I've addressed the issue with frames. I cannot move away from them > quickly, because I don't control this. It was carefully explained why, > in another posting.
Sites with framesets present extra complexity and difficulties, but it is *not* impossible for a properly-coded frameset website to be crawled and indexed.
Removing framesets would be the best solution but if the framesets are done properly, sites can be crawlable and indexable. Why can't your shop-software-provider help?
As you say yourself, many other Danish shops use the same software successfully.
Can you please give some specific URLs for these shops so that we can see the difference/similarity?
> (By February, I'm forced out of business, so only Google can really > help me now).
I sincerely hope that you will listen to highly-respected contributors like SoftPlus (a very skilled, knowledgeable, patient and helpful person - that I have never met!) and follow at least some of the advice - or at least do not wait for Google to take over your web design and application-serving.
And please don't keep giving insults to the good folk, such as SoftPlus, who are trying to help, even if you cannot agree with what they say. Hey, what do they know? It's not as though they do this sort of thing for a living, is it?
[ If SoftPlus gives you advice about furniture ... that would be different. :-) ]
You said:
> - The DOCTYPE is not the problem.
Of course, obviously Google can crawl pages without a DOCTYPE. The point is that declaring the DOCTYPE is not only correct but ensures that your page is checked against the appropriate standard during validation. If you do not declare, a default is assumed and this may or may not be the one that you intended or was assumed by the page-author.
> Matt Cutts specifically said that W3 validation isn't important and not worth worrying about. Google is used to pages of varying quality and I'm sure that's not the reason why my shop got excluded. Wouldn't you agree?
I would agree that, as I have said in this forum in the past, 100% correct w3 validation is neither necessary nor sufficient for Google crawling and indexing. It all depends on what the error is.
w3 validation is a useful diagnostic tool (one of many).
In his first video, Matt Cutts said "The number 1 thing that most people make mistakes on SEO ... is that they don't make their sites crawlable." The w3 validator is very good at detecting badly-formed head sections that will prevent crawling.
If someone said to you "My car is faulty. Am I safe to drive from Copenhagen to Hamburg?" what would you advise? Would you say "No, your car must be 100% perfect" or would you say "Yes, it's fine, go ahead, no problem." I think a sensible answer would be, "Well, it depends on exactly what is wrong with the car! Let's take a look at it ... "
And you also need to listen to Matt's video very carefully - he is talking primarily about "accessibility" and this is a different issue from "crawlability". Checking for "accessibility" is only done AFTER ensuring that the site's HTML is valid.
Matt Cutts (in his video) says that having an "accessible" website (which passes w3 accessibility validator) is not an important factor in Google indexing. "Accessibility" means that the site is optimised for use by everyone including people with disabilities, such as blindness, deafness, etc.
> What you say about ID= I would like to know more about. Is it a fact? > Where have you seen that?
This *was* official Google guidance until recently. The exact words were: "Don't use "&id=" as a parameter in your URLs, as we don't include these pages in our index." But on 25 October 2006, Googler Vanessa Fox said (in Official Google Webmaster Central Blog): "However, we've recently removed that technical guideline, ..."
So I am sorry that my suggestion was out of date. This requirement does not apply now. Thanks for challenging that!
> I'm not alone with this problem. 5 shops were excluded.
It's clear I need to give an update here, if only to save your sanity, my sanity, and the sanity of people who have been subjected to your not-malicious but still hugely-numbering posts here in every topic.
1) We don't take punitive action based upon e-mail we receive or postings made here.
2) To my knowledge, few, if any Googlers are in the offices today, at least in those offices that deal with Search Quality. Fri-Tue is a holiday time for us. This means that few of us are reading our work mail and these forums, as you can understand.
3) I have already asked some colleagues to look into your site's situation, much as I've done for many other sites here in similar situations. But, as per #2, it's unlikely that things will be investigated prior to Wednesday. Please, please, please... note that posting here more times will not expedite matters. Working from home today, even I don't have ready access to all my investigative tools.
4) Sussie... I am going to have to be blunt: at least from what I've been able to tell, it seems clear that you aren't doing anything "evil," but given some of the issues pointed out to you in various threads (and in particular the frames stuff), it's not surprising to me that you've tripped algorithms that've seemingly resulted in poor indexing and ranking for your site. As I noted, I've asked some engineers to take a look at the situation, but for the long haul, I urge you to do whatever you can to make your site more broadly accessible (which likely means re-evaluating the frames) and so on.
5) I am likely to delete many of the posts you've made recently. Those Googlers and others who are away for this time are unlikely to see any of them, and those Googlers and others who are here and reading this site have likely already seen them and begun researching (and frankly would have likely seen just one or two key posts as well ;-). I sympathize with your situation, but also must keep this board sane for the thousands of other Webmasters who rely upon this board not being an anarchic flood of repeated help pleas.
P.S. -- I can assure you that your site issues have *nothing* to do with Sitemaps or Webmaster Tools. Despite others' conspiracy theories, it'd be like blaming the doctor for telling you you have a problem. Making a sitemap helps us better understand your site; using Webmaster Tools helps YOU better understand how we see your site.
A consistent active Google presence would likely be a better route to making this forum more useful and sane.
When the people who are getting the most Google response are the most irritating and intrusive posters, you are likely setting a precedent that you will come to regret.
It's a serious e-mail. Thank you. My response is not meant to start a new series of posts. It's solely my response to your points.
1: Right. I think it was a misunderstanding. The webmaster tools said my pages weren't indexed, which I took as an exclusion (cause it looked just like it). Other friendly people in here pointed out that they saw the same in their webmaster account and that (even though it looks wierd), isn't and exclusion.
2: No problem with the holiday. I just wanted to know that I'm not left to my own devices and that my point got across to someone (time remains to be of the essense for my shops survival though).
3: Sorry. Your post were only just now brought to my attention and it's so easy to miss things in the web interface (especially when you haven't slept and is on stress hormone).
4: Surely the frames need to go. Noone deliberately wanted these problems, but you know how systems develop. The shop system provider is working on it, but it takes time and some kind of solution for a framed site would be good to have. As you have probably read, I aleady missed Christmas in my shop (together with the other shops that were excluded), so waiting now for a non-framed site would kill my business. I'm a little edgy about that. I also don't need perfect ranking, but being totally gone from Google (with the exception of my front-page) just is a little extreme. To make matters worse, my competitor now have #1 rankings on everything, even though their site is on the same shop provider as mine.
5: Quite alright with me. Please accept my plea for forgiveness. You have probably tried to get support from large corporations yourself and know how it is. Try or die. I had to try.
I'm looking forward to hearing more about our problems here and getting things fixed to the degree it's possible. Thanks again for your assistance and continued attention.
> It's clear I need to give an update here, if only to save your sanity, > my sanity, and the sanity of people who have been subjected to your > not-malicious but still hugely-numbering posts here in every topic.
> 1) We don't take punitive action based upon e-mail we receive or > postings made here.
> 2) To my knowledge, few, if any Googlers are in the offices today, at > least in those offices that deal with Search Quality. Fri-Tue is a > holiday time for us. This means that few of us are reading our work > mail and these forums, as you can understand.
> 3) I have already asked some colleagues to look into your site's > situation, much as I've done for many other sites here in similar > situations. But, as per #2, it's unlikely that things will be > investigated prior to Wednesday. Please, please, please... note that > posting here more times will not expedite matters. Working from home > today, even I don't have ready access to all my investigative tools.
> 4) Sussie... I am going to have to be blunt: at least from what I've > been able to tell, it seems clear that you aren't doing anything > "evil," but given some of the issues pointed out to you in various > threads (and in particular the frames stuff), it's not surprising to me > that you've tripped algorithms that've seemingly resulted in poor > indexing and ranking for your site. As I noted, I've asked some > engineers to take a look at the situation, but for the long haul, I > urge you to do whatever you can to make your site more broadly > accessible (which likely means re-evaluating the frames) and so on.
> 5) I am likely to delete many of the posts you've made recently. Those > Googlers and others who are away for this time are unlikely to see any > of them, and those Googlers and others who are here and reading this > site have likely already seen them and begun researching (and frankly > would have likely seen just one or two key posts as well ;-). I > sympathize with your situation, but also must keep this board sane for > the thousands of other Webmasters who rely upon this board not being an > anarchic flood of repeated help pleas.
Forgive me again for being direct in my reply. It's not to be insensitive to your arguments or hard-earned exprience. I'm merely challenge your opinions, weigh them against my experience, and then for the rest I tell you my honest opinion. When you provide the missing puzzle, you can be sure that I won't overlook it.
> Sussie, what matters to Google is what they found when the Googlebot > visited your site.
Yes, I know that it's only what Google sees that matters.
Do not mistake me for a beginner. I'm a fully educated programmer (C, ASM, Java, PHP, Cobol, Pascal, etc) and I've worked with all sorts of IT, on and off, since I was 12 years old. I've been involved in many projects over time and business web-sites as well. I do 3D, Flash, and I've designed furniture from scratch that way (which I've then had produced and sold worldwide). I've also worked as a webmaster in a large Danish corporation and I have a number of friends that are among Denmarks finest when it comes to IT knowledge. My knowledge and experience bank is pretty big, but I just didn't expect to use it here (since I wanted to focus on furniture). A few notable personalities from my small area is the inventor of C++ and the creator of Skype.
Enough about that.. I'm in furniture now and that's my personal interest, and where I've invested my time and money for the past long while.
Back to your comments: Google finds the frameset front page and that's not a problem, since I have a static front page on the top level domain that Google indexes just fine and crawls okay as well. It's only this static frontpage that will lead Google to the other pages. This part works.
> (On a very minor point, I also notice that 4 of these 6 pages have the > same head title.)
Perfectly normal and not a violation of any guideline that I've heard about. All my product pages have a unique title, just like it should be.
> www.casanovafurniture.dk it is visible but it contains many HUNDREDS of > links (Google limit 100) and many HUNDREDS of images (severe bandwidth
There is another link.. Try the bottom left one on www.casanovafurniture.dk. On that page there is only 50 links per page (to be on the safe side).
The other sitemap is a static one I made, as a test - because Matt Cutts say in his videos that dynamic pages that link to dynamic pages that link to dynamic pages can be bad (on some other search engines:). For that reason, I put in a bunch of static links.
Google's *limit* is not 100. The guideline says exactly this:
- Offer a site map to your users with links that point to the important parts of your site. If the site map is larger than 100 or so links, you may want to break the site map into separate pages.
So why not be on the safe side? Well, because this is not the problem that keeps all our produts away from google. There are external links to many products too, but they too don't get indexed. Let's focus on the big issues first, and then we can fine-tune the number of links later on.
Ironically, as I write this, the big static page just got into Google's index. My dynamic sitemap with the ? character in it, didn't get indexed yet. I won't draw any major conclusions from this, except it's a step in the right direction. No products are indexed yet.
> Sites with framesets present extra complexity and difficulties, but it > is *not* impossible for a properly-coded frameset website to be crawled > and indexed.
How?
> Removing framesets would be the best solution but if the framesets are > done properly, sites can be crawlable and indexable. Why can't your > shop-software-provider help?
They are helping (now), but it's a VERY big system. All their modules, the shopping cart, their templates, the documentation, the testing with hundreds of shops, etc. We are only 5 shops with trouble and what is the company to do (that they aren't already)? A big company doesn't understand Google better than you and I. Difficult balance to know how many resources to poor into Google, when it's not an exact science. I like Google for the same reason. It's kind of a romantic component, but I don't like when it completely rejects me and ruins my business (without any obvious reason or counter-measures).
> I sincerely hope that you will listen to highly-respected contributors > like SoftPlus (a very skilled, knowledgeable, patient and helpful > person - that I have never met!) and follow at least some of the advice > - or at least do not wait for Google to take over your web design and > application-serving.
Oh of course. SoftPlus is great and I've taken his advice to heart. Softplus saw things from one angle, and as the owner of this business, I just had a few more things to consider. All the times that SoftPlus said go "no-frames" was a waste of my time, since I knew it already and had started clearly that it wasn't an option right now. Once we got tuned into each other and actually listened, we were fine.
> And please don't keep giving insults to the good folk, such as > SoftPlus, who are trying to help, even if you cannot agree with what > they say. Hey, what do they know? It's not as though they do this > sort of thing for a living, is it?
I've not deliberately insulted anyone, but kept my posts well-founded. Some might take offense when I outrule whatever they say, but that's really not smart. I'm of course also challenging the "experts", cause we all have opinions and we can all read the same information from Google. The whole point is that Google don't want a number of experts. Google want their algoritms to be unpredictable and they are handling that just fine IMO. Of course, when businesses like mine are totally booted by Google, then that's bad and I think that justifies somewhat my irrational behavior in here. I think Google should offer a paid hotline for whitehat businesses that get excluded without having done evil. It would allow Google to take even stronger measures against true spammers.
Also, in regards to insults, then see what I received in private e-mail about 1 hour ago:
"First, let me say I know you must be deeply frustrated by the turn of events that has so dramatically affected your business. I'm replying to you directly rather than by public forum simply because I've expressed my views in the past in these forums and have no further interest in wading through all the verbiage that gets thrown up by "experts" who frequently haven't a clue what they are talking about."
It echoes part of what I've been saying too... But I'm really not here to discuss this. I'm here to be productive and get verified/important helpful data - to save my business.
> Of course, obviously Google can crawl pages without a DOCTYPE.
For now, this is enough for me!
Some of the best indexed sites I know, completely bypassed stuff like DOCTYPE, meta tags, W3 validation, etc. This is my experience. Spending my last days in business on updating the DOCTYPE on a few pages and having the shop provider removing the warnings spit out by a W3 validator would be less than optimal. I have to address the bigger issues first and then have my shop provider change these details when they have time. It's not going to have any particular effect on the ranking by Google (I'm very sure about that, since that is an area where I personally have some experience and it's backed up generally by the highly esteemed Google crew).
> So I am sorry that my suggestion was out of date. This requirement does > not apply now. Thanks for challenging that!
Well, that's how results are made. Thanks for not feeling offended by me challenging the ideas thrown at me.
> > I'm not alone with this problem. 5 shops were excluded. > Do you know what they are doing about it?
Yes... I'm in contact with these shops.
Most of them are actually waiting for me to come up with a solution and further, few of them have the technical insight to engage in meaningful discussions in a forum like this. We have our own support forum at the shop provider, where I've updated the other shops with my findings and put pressure on the shop provider too. After a bit of discussion, to prove that I'm serious and not a beginner, the shop provider has now ensured me their full co-operation. The discussions are online, but in Danish :) With Google in the game too now, I hope that a major crisis can be softened.
Ask your shop provider to add a "target=_top" tag to each product link so that they open in a new page and have a "Back" link on each of them. It may be annoying to the user but effective for having the products pages indexed till a final solution is reached. Still, that shop has too many frames to ever make it operative.
Just something to consider... Others may be more willing to help if:
1. you can keep your inquiries strictly to the point. 2. you remember that members who donate their time and specially high level Google employees may not be able to go through lengthy posts. 3. you can refrain from annoying others. 4. having some perspective: there may be thousands in your same situation or worse.
|
