I'd like to make a sitemap but I can't seem to get the site crawled
with the program, other than robots.txt, all I get in URL list is one
URL (the main url). www.mediaportal.hr
I read on this a post with a similar problem on this board but it
doesn't help solve the problem.
I set up the program through it's wizard but it doesnt seem to crawl
anything (crawlers remain idle). any ideas? help please!
The when you request a crawl use the down arrow next to the button Re-
crawl and pikc This site.
Give it plenty of time to cawl, your site is big and/or slow for
crawlers (dont' knwo whihc, only tested a bit and seems to take long).
If your site has any redirecittions on it as it's being crawled that's
going to cause problems.
You can crawl your site using Xenu from
http://home.snafu.de/tilman/xenulink.html and see if there are no errors (404, redirections, etc) during
navigation. These will need to be fixed before you can hope to build a
sitemap using Gsietcrawler or any other tool.
On May 21, 4:34 pm, prodigy2006 <adever...@gmail.com> wrote:
> I'd like to make a sitemap but I can't seem to get the site crawled
> with the program, other than robots.txt, all I get in URL list is one
> URL (the main url).www.mediaportal.hr
> I read on this a post with a similar problem on this board but it
> doesn't help solve the problem.
> I set up the program through it's wizard but it doesnt seem to crawl
> anything (crawlers remain idle). any ideas? help please!
Once it finishes (maybe 1 or 2 mintes for only 28 urls) it will say
crawlers are emtpy, idle.
You can check what has bene found by clicking URL List and refresh. It
shoudl show the list of urls it has found.
Then click Generate > Google Sitemap
And so on.
On Jun 28, 8:34 pm, Jazbo <wellspri...@gmail.com> wrote:
Re-crawl this site from the top toolbar, or the url list? The only
option on the top is recrawl this project.
I understand the final part, I made and uploaded a Sitemap from the
one url. But it seems like no matter what I do, it seems like I can't
get it to crawl...
Thanks for getting back so soon!
On Jun 28, 5:55 pm, webado2 <web...@gmail.com> wrote:
> Once it finishes (maybe 1 or 2 mintes for only 28 urls) it will say
> crawlers are emtpy, idle.
> You can check what has bene found by clicking URL List and refresh. It
> shoudl show the list of urls it has found.
> Then click Generate > Google Sitemap
> And so on.
> On Jun 28, 8:34 pm, Jazbo <wellspri...@gmail.com> wrote:
> > I am having the same problem as prodigy2006. I am using Windows XP
> > pro.
> > I ran the url in Xenu and had no errors and only 28 links...
----- Original Message ----- From: "Jazbo" <wellspri...@gmail.com>
To: "SOFTplus GSiteCrawler" <gsitecrawler@googlegroups.com>
Sent: Sunday, June 28, 2009 9:14 PM
Subject: [GSiteCrawler] Re: crawler idle
Re-crawl this site from the top toolbar, or the url list? The only
option on the top is recrawl this project.
I understand the final part, I made and uploaded a Sitemap from the
one url. But it seems like no matter what I do, it seems like I can't
get it to crawl...
Thanks for getting back so soon!
On Jun 28, 5:55 pm, webado2 <web...@gmail.com> wrote:
> Did you click Re-crawl > This Site ?
> Once it finishes (maybe 1 or 2 mintes for only 28 urls) it will say
> crawlers are emtpy, idle.
> You can check what has bene found by clicking URL List and refresh. It
> shoudl show the list of urls it has found.
> Then click Generate > Google Sitemap
> And so on.
> On Jun 28, 8:34 pm, Jazbo <wellspri...@gmail.com> wrote:
> > I am having the same problem as prodigy2006. I am using Windows XP
> > pro.
> > I ran the url in Xenu and had no errors and only 28 links...
I don't understand it, I've tried a few other sites, and they worked
with no problem. It must be an issue with the site? Why would all of
the links test with no problems and still not have it crawl? Thanks
for the help!
On Jun 28, 7:58 pm, Jazbo <wellspri...@gmail.com> wrote:
So either you fix all your internal links to use all www urls or run
the crawler for http://scentimentsfromtheheart.com/ and submit the
site that way, without www.
On Jun 29, 5:08 pm, Jazbo <wellspri...@gmail.com> wrote:
> I don't understand it, I've tried a few other sites, and they worked
> with no problem. It must be an issue with the site? Why would all of
> the links test with no problems and still not have it crawl? Thanks
> for the help!
> On Jun 28, 7:58 pm, Jazbo <wellspri...@gmail.com> wrote:
> > It's not fully functional yet, but the pages work. Here it is:
Actually it's worse. In addition to not staying on one particular
domain, you are also generating urls with some kind of session id.
Sort of never ending.
You have some broken links, and some redirected ones.
Sometimes somethign stupid like an opening comment like <!-- which
doesn't get closed will mean that everything from that place one is
ignored. Thus no further links are found.
I don't know yet.
Fix what you can from the site and try again.
On Jun 29, 5:08 pm, Jazbo <wellspri...@gmail.com> wrote:
> I don't understand it, I've tried a few other sites, and they worked
> with no problem. It must be an issue with the site? Why would all of
> the links test with no problems and still not have it crawl? Thanks
> for the help!
> On Jun 28, 7:58 pm, Jazbo <wellspri...@gmail.com> wrote:
> > It's not fully functional yet, but the pages work. Here it is: