Error at path_handler.copy while creating a node
51 views
Skip to first unread message
Lars Bähren
Feb 22, 2015, 3:42:45 AM2/22/15
to webgen...@googlegroups.com
I continue running into the following error when trying to generate the website:
The interesting bit about this is, that the page source did not change at all - I did edit another page though, hence the rebuild. The page in question has the following metadata section:
From the Webgen documentation I understand that indeed I can use the routed_title to avoid collisions - going through the various pages there is no second page which would result in the same acln (which is good, I think). I would expect there to be an entry in the cache from the previous run: instead of recognizing the page (and skipping generation) I get an error.
I can circumvent the problem (and get the site generated) when removing tmp/webgen.cache but the side-effect of this (of course) is that everything is generated anew (which then takes 10+ minutes).
INFO Generating website...
INFO [update] </blog/>
INFO [update] </blog/2003.html>
webgen encountered a problem:
Error at path_handler.copy while creating a node from </blog/2003.en.html#blog-entries--2003>:
Can't have two nodes with same alcn: /blog/2003.en.html#blog-entries--2003
make[2]: *** [CMakeFiles/Website] Error 1
make[1]: *** [CMakeFiles/Website.dir/all] Error 2
make: *** [all] Error 2The interesting bit about this is, that the page source did not change at all - I did edit another page though, hence the rebuild. The page in question has the following metadata section:
---
title: "2003"
routed_title: "Blog entries 2003"
in_menu: false
---From the Webgen documentation I understand that indeed I can use the routed_title to avoid collisions - going through the various pages there is no second page which would result in the same acln (which is good, I think). I would expect there to be an entry in the cache from the previous run: instead of recognizing the page (and skipping generation) I get an error.
I can circumvent the problem (and get the site generated) when removing tmp/webgen.cache but the side-effect of this (of course) is that everything is generated anew (which then takes 10+ minutes).
Thomas Leitner
Feb 22, 2015, 5:29:35 AM2/22/15
to webgen...@googlegroups.com
Hi,
On 2015-02-22 00:42 -0800 Lars Bähren wrote:
> I continue running into the following error when trying to generate
> the website:
>
> INFO Generating website...
> INFO [update] </blog/>
> INFO [update] </blog/2003.html>
> webgen encountered a problem:
> Error at path_handler.copy while creating a node from
> </blog/2003.en.html #blog-entries--2003>:
> Can't have two nodes with same alcn:
> /blog/2003.en.html#blog-entries--2003
> make[2]: *** [CMakeFiles/Website] Error 1
> make[1]: *** [CMakeFiles/Website.dir/all] Error 2
> make: *** [all] Error 2
>
> The interesting bit about this is, that the page source did not
> change at all - I did edit another page though, hence the rebuild.
Every time a page gets written and this page has the "fragments"
content processor enabled, fragment nodes (i.e. the ones for the
in-page header anchors) for the in-page links are created. However, if
an equivalent node already exists (e.g. from a cached previous run) this
node should normally be reused.
There seems to be a bug somewhere in that logic.
Can you create a small example site that reproduces the bug? Or are the
source for you website publicly available?
> The page in question has the following metadata section:
>
> ---
> title: "2003"
> routed_title: "Blog entries 2003"
> in_menu: false
> ---
>
> From the Webgen documentation I understand that indeed I can use the
> routed_title to avoid collisions - going through the various pages
> there is no second page which would result in the same acln (which is
> good, I think). I would expect there to be an entry in the cache from
> the previous run: instead of recognizing the page (and skipping
> generation) I get an error.
The alcn are created from the path names themselves, the title doesn't
come into play.
The routed_title meta information is mainly used in index files for
directories to specify a localized directory name. I.e. when a link to
the directory should be rendered, the routed_title meta information is
looked up in the localized index page and if not found there, the
directory title itself is used.
> I can circumvent the problem (and get the site generated) when
> removing *tmp/webgen.cache* but the side-effect of this (of course)
to reproduce this somehow.
I tried reproducing this in the source for the webgen documentation
website but failed so far.
What may also help is if you send me the debug output (webgen --debug)
of the main run (ie. one with no cache file) and then the debug output
of the run where the error happens.
Best regards,
Thomas
On 2015-02-22 00:42 -0800 Lars Bähren wrote:
> I continue running into the following error when trying to generate
> the website:
>
> INFO Generating website...
> INFO [update] </blog/>
> INFO [update] </blog/2003.html>
> webgen encountered a problem:
> Error at path_handler.copy while creating a node from
> </blog/2003.en.html #blog-entries--2003>:
> Can't have two nodes with same alcn:
> /blog/2003.en.html#blog-entries--2003
> make[2]: *** [CMakeFiles/Website] Error 1
> make[1]: *** [CMakeFiles/Website.dir/all] Error 2
> make: *** [all] Error 2
>
> The interesting bit about this is, that the page source did not
> change at all - I did edit another page though, hence the rebuild.
content processor enabled, fragment nodes (i.e. the ones for the
in-page header anchors) for the in-page links are created. However, if
an equivalent node already exists (e.g. from a cached previous run) this
node should normally be reused.
There seems to be a bug somewhere in that logic.
Can you create a small example site that reproduces the bug? Or are the
source for you website publicly available?
> The page in question has the following metadata section:
>
> ---
> title: "2003"
> routed_title: "Blog entries 2003"
> in_menu: false
> ---
>
> From the Webgen documentation I understand that indeed I can use the
> routed_title to avoid collisions - going through the various pages
> there is no second page which would result in the same acln (which is
> good, I think). I would expect there to be an entry in the cache from
> the previous run: instead of recognizing the page (and skipping
> generation) I get an error.
come into play.
The routed_title meta information is mainly used in index files for
directories to specify a localized directory name. I.e. when a link to
the directory should be rendered, the routed_title meta information is
looked up in the localized index page and if not found there, the
directory title itself is used.
> I can circumvent the problem (and get the site generated) when
> is that everything is generated anew (which then takes 10+ minutes).
That is certainly not satisfactory. However, to fix this I first need
to reproduce this somehow.
I tried reproducing this in the source for the webgen documentation
website but failed so far.
What may also help is if you send me the debug output (webgen --debug)
of the main run (ie. one with no cache file) and then the debug output
of the run where the error happens.
Best regards,
Thomas
Lars Bähren
Feb 22, 2015, 5:48:09 AM2/22/15
to webgen...@googlegroups.com
On Sunday, February 22, 2015 at 11:29:35 AM UTC+1, Thomas Leitner wrote:
Every time a page gets written and this page has the "fragments"
content processor enabled, fragment nodes (i.e. the ones for the
in-page header anchors) for the in-page links are created. However, if
an equivalent node already exists (e.g. from a cached previous run) this
node should normally be reused.
Hello Thomas, this is what I would expect from cache - otherwise a comparison would not be possible across subsequent generation runs.
There seems to be a bug somewhere in that logic.
This probably is something you can judge far better than me.
Can you create a small example site that reproduces the bug? Or are the
source for you website publicly available?
Sure thing: the full source (pages and surrounding scripts) are available via Github - sources for the blog pages are in src/pages/blog
That is certainly not satisfactory. However, to fix this I first need
to reproduce this somehow.
Sounds reasonable.
I tried reproducing this in the source for the webgen documentation
website but failed so far.
What may also help is if you send me the debug output (webgen --debug)
of the main run (ie. one with no cache file) and then the debug output
of the run where the error happens.
Ok, will re-run this with with debugging enabled...
Cheers,
Lars
Lars Bähren
Feb 22, 2015, 11:01:57 AM2/22/15
to webgen...@googlegroups.com
On Sunday, February 22, 2015 at 11:29:35 AM UTC+1, Thomas Leitner wrote:
What may also help is if you send me the debug output (webgen --debug)
of the main run (ie. one with no cache file) and then the debug output
of the run where the error happens.
Hi Thomas,
sorry for the slight delay, but below find the output from the above suggested configuration (I actually added a further "-v")
INFO Populating node tree took 30.58 seconds
INFO [update] </blog/>
INFO [timing] </blog/> rendered in 0.00 seconds
DEBUG Using default template in language 'en' for </blog/2003.en.html>
DEBUG Using default template in language 'en' for </default.template>
INFO [update] </blog/2003.html>
DEBUG Replacing tag lang with data nil and body '' in </default.template>
DEBUG Replacing tag lang with data nil and body '' in </default.template>
DEBUG Replacing tag relocatable with data "andreas01.css" and body '' in </default.template>
DEBUG Replacing tag relocatable with data "print.css" and body '' in </default.template>
DEBUG Replacing tag title with data nil and body '' in </default.template>
DEBUG Replacing tag relocatable with data "/index.html" and body '' in </default.template>
DEBUG Replacing tag menu with data {"options"=>{"mi"=>{"in_menu"=>true}, "sort"=>true, "siblings"=>[0, -1]}} and body '' in </default.template>
DEBUG Using default template in language '' for </templates/tag.template>
DEBUG Using default template in language '' for </default.template>
DEBUG Replacing tag menu with data {"options"=>{"sort"=>true, "absolute_levels"=>[0, 2], "mi"=>{"in_menu"=>true}}} and body '' in </default.template>
DEBUG Replacing tag relocatable with data "/blog/index.html" and body '' in </default.template>
DEBUG Replacing tag breadcrumb_trail with data nil and body '' in </default.template>
DEBUG Replacing tag date with data {"format"=>"%Y"} and body '' in </default.template>
DEBUG Replacing tag title with data nil and body '' in </blog/2003.en.html>
DEBUG Creating node version 'default' from path </blog/2003.en.html#blog-entries--2003> with copy handler
webgen encountered a problem:
Error at path_handler.copy while creating a node from </blog/2003.en.html#blog-entries--2003>:
Can't have two nodes with same alcn: /blog/2003.en.html#blog-entries--2003
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/tree.rb:103:in `register_node'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/node.rb:99:in `initialize'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler/base.rb:135:in `new'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler/base.rb:135:in `create_node'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler/copy.rb:32:in `create_nodes'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler.rb:401:in `block in create_nodes_with_path_handler'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler.rb:392:in `each'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler.rb:392:in `map'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler.rb:392:in `create_nodes_with_path_handler'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler.rb:353:in `create_secondary_nodes'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/content_processor/fragments.rb:73:in `block in create_fragment_nodes'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/content_processor/fragments.rb:65:in `each'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/content_processor/fragments.rb:65:in `create_fragment_nodes'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/content_processor/fragments.rb:18:in `call'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/content_processor.rb:104:in `call'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler/page_utils.rb:46:in `block in render_block'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler/page_utils.rb:45:in `each'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler/page_utils.rb:45:in `render_block'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/content_processor/blocks.rb:89:in `render_block'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/content_processor/blocks.rb:26:in `block in call'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/content_processor/blocks.rb:21:in `gsub!'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/content_processor/blocks.rb:21:in `call'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/content_processor.rb:104:in `call'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler/page_utils.rb:46:in `block in render_block'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler/page_utils.rb:45:in `each'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler/page_utils.rb:45:in `render_block'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler/page.rb:28:in `content'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler/base.rb:66:in `content'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler.rb:307:in `block in write_node'
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/benchmark.rb:281:in `measure'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler.rb:307:in `write_node'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler.rb:280:in `block in write_tree'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler.rb:271:in `each'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/path_handler.rb:271:in `write_tree'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/task/generate_website.rb:23:in `block in call'
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/benchmark.rb:281:in `measure'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/task/generate_website.rb:20:in `call'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/task.rb:89:in `execute'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/website.rb:209:in `execute_task'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/cli/commands/generate.rb:29:in `execute'
/Library/Ruby/Gems/2.0.0/gems/cmdparse-2.0.6/lib/cmdparse.rb:464:in `parse'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/lib/webgen/cli.rb:192:in `parse'
/Library/Ruby/Gems/2.0.0/gems/webgen-1.2.1/bin/webgen:5:in `<top (required)>'
/usr/bin/webgen:23:in `load'
/usr/bin/webgen:23:in `<main>'
Scenario is the same as described earlier: I have been updating 'blog/index.page' and 'blog/2015.page' - but the error I am getting originates from 'blog/2003.page' which was not touched.
Cheers,
Lars
Thomas Leitner
Feb 22, 2015, 5:22:26 PM2/22/15
to webgen...@googlegroups.com
On 2015-02-22 08:01 -0800 Lars Bähren wrote:
> Scenario is the same as described earlier: I have been updating
> 'blog/index.page' and 'blog/2015.page' - but the error I am getting
> originates from 'blog/2003.page' which was not touched.
Thanks for the link to the sources and the explanation on how to
> Scenario is the same as described earlier: I have been updating
> 'blog/index.page' and 'blog/2015.page' - but the error I am getting
> originates from 'blog/2003.page' which was not touched.
reproduce the bug! I think that I have located the problem and I'm
working on a fix.
-- Thomas
Lars Bähren
Feb 22, 2015, 10:37:12 PM2/22/15
to webgen...@googlegroups.com
On Sunday, February 22, 2015 at 11:22:26 PM UTC+1, Thomas Leitner wrote:
Thanks for the link to the sources and the explanation on how to
reproduce the bug! I think that I have located the problem and I'm
working on a fix.
Hi Thomas - good look with the fix, looking forward to pull an update once this is resolved.
- Lars
Thomas Leitner
Feb 23, 2015, 4:11:14 AM2/23/15
to webgen...@googlegroups.com
Hi Lars,
I have fixed the problem and it works now with your website sources.
The fix is on Github. Please verify and if this resolves your problem I
will release a new version.
Please make sure that you delete all webgen caches before you try it
out because the source path recognition had to be changed to fix this.
The problem was that a path like "2015.en.html" was handled as
"en.html" with "en" being the basename and not the language part of the
path which lead to collisions.
Thanks for reporting this bug!
Regarding something else: Yours is probably the biggest website made
with webgen I have encountered so far. The webgen documentation website
is more complex in its use of webgen but your site with 4730 page files
is by far bigger. I will see if I can optimize webgen internals so that
it runs faster.
Cheers,
Thomas
The fix is on Github. Please verify and if this resolves your problem I
will release a new version.
Please make sure that you delete all webgen caches before you try it
out because the source path recognition had to be changed to fix this.
The problem was that a path like "2015.en.html" was handled as
"en.html" with "en" being the basename and not the language part of the
path which lead to collisions.
Thanks for reporting this bug!
Regarding something else: Yours is probably the biggest website made
with webgen I have encountered so far. The webgen documentation website
is more complex in its use of webgen but your site with 4730 page files
is by far bigger. I will see if I can optimize webgen internals so that
it runs faster.
Cheers,
Thomas
Thomas Leitner
Feb 23, 2015, 11:55:34 AM2/23/15
to webgen...@googlegroups.com
On 2015-02-22 19:37 -0800 Lars Bähren wrote:
>
>
>
>
I still need to further investigate on how to optimize it.
However, what I have found is that your making webgen do much more work
than necessary. Since the menu statement at
https://github.com/lbaehren/Website/blob/master/ext/templates/data/lars01/default.template#L59
is commented out, the result will not appear on the website itself but
webgen still processes it, creating two menus instead of one.
After commenting out the menu (just place a backslash in front of it,
i.e. \{menu:...}), I cut the first time generation down by about 66%
(from 300 seconds to 100 seconds on my machine).
Cheers,
Thomas
Lars Bähren
Feb 24, 2015, 2:18:00 AM2/24/15
to webgen...@googlegroups.com
On Monday, February 23, 2015 at 5:55:34 PM UTC+1, Thomas Leitner wrote:
On 2015-02-22 19:37 -0800 Lars Bähren wrote:
>
>
> On Sunday, February 22, 2015 at 11:22:26 PM UTC+1, Thomas Leitner
> wrote:
> >
> > Thanks for the link to the sources and the explanation on how to
> > reproduce the bug! I think that I have located the problem and I'm
> > working on a fix.
> >
>
> Hi Thomas - good look with the fix, looking forward to pull an update
> once this is resolved.
Hi Thomas,
I now (finally) managed to get the update installed and running. With this after adding a new blog entry Webgen correctly generates only the changes/new pages.
INFO Generating website...
INFO [update] </blog/>
INFO [update] </blog/2015.html>
INFO [update] </blog/2015/01/index.html>
INFO [update] </blog/2015/02/>
INFO [create] </blog/2015/02/2015-02-24_07-59.html>
INFO [update] </blog/2015/02/index.html>
INFO [update] </blog/2015/index.html>
INFO [update] </blog/index.html>
INFO [update] </blog/upcoming/>
INFO [update] </blog/upcoming/index.html>
INFO ... done in 477.85 secondsGreat work - thanks a lot.
I have found what is the bottleneck during website (re-)generation but
I still need to further investigate on how to optimize it.
However, what I have found is that your making webgen do much more work
than necessary. Since the menu statement at
https://github.com/lbaehren/Website/blob/master/ext/templates/data/lars01/default.template#L59
is commented out, the result will not appear on the website itself but
webgen still processes it, creating two menus instead of one.
After commenting out the menu (just place a backslash in front of it,
i.e. \{menu:...}), I cut the first time generation down by about 66%
(from 300 seconds to 100 seconds on my machine).
Thanks for the tip - that definitely is something I was not aware of. I will give this a try as well.
Cheersm
Lars
Lars Bähren
Feb 24, 2015, 2:28:47 AM2/24/15
to webgen...@googlegroups.com
On Monday, February 23, 2015 at 5:55:34 PM UTC+1, Thomas Leitner wrote:
After commenting out the menu (just place a backslash in front of it,
i.e. \{menu:...}), I cut the first time generation down by about 66%
(from 300 seconds to 100 seconds on my machine).
Wow, that is quite impressive. I put in the changes you suggested - with the next generation pass of course the complete site would be rendered... but this ran faster than the previous partial rebuild!
[100%] Rendering website from sources ...
INFO Generating website...
INFO [update] </blog/>
INFO [update] </blog/2003.html>
INFO [update] </blog/2003/2003-11/2003-11-03_20-47.html>
INFO [update] </blog/2003/2003-11/2003-11-03_20-54.html>
INFO [update] </blog/2003/2003-11/2003-11-03_20-57.html>
...
INFO [update] </writing/book_reviews/moondial.html>
INFO [update] </writing/book_reviews/red_rabbit.html>
INFO [update] </writing/book_reviews/the_divine_secrets_of_the_ya-ya-sisterhood.html>
INFO [update] </writing/book_reviews/the_last_year_of_being_married.html>
INFO [update] </writing/book_reviews/the_undutchables.html>
INFO [update] </writing/index.html>
INFO ... done in 256.40 seconds
[100%] Built target WebsiteI'll make a few local changes to get some numbers on the typical scenario of adding a new blog entry - but this already is impressive as is!
Cheersm
Lars
Thomas Leitner
Feb 24, 2015, 3:49:32 AM2/24/15
to webgen...@googlegroups.com
On 2015-02-23 23:28 -0800 Lars Bähren wrote:
> On Monday, February 23, 2015 at 5:55:34 PM UTC+1, Thomas Leitner
> wrote:
>
> After commenting out the menu (just place a backslash in front of it,
> > i.e. \{menu:...}), I cut the first time generation down by about
> > 66% (from 300 seconds to 100 seconds on my machine).
> >
>
> Wow, that is quite impressive. I put in the changes you suggested -
> with the next generation pass of course the complete site would be
> rendered... but this ran faster than the previous partial rebuild!
Yeah, the reason why this works on your website is that the statement
> On Monday, February 23, 2015 at 5:55:34 PM UTC+1, Thomas Leitner
> wrote:
>
> After commenting out the menu (just place a backslash in front of it,
> > i.e. \{menu:...}), I cut the first time generation down by about
> > 66% (from 300 seconds to 100 seconds on my machine).
> >
>
> Wow, that is quite impressive. I put in the changes you suggested -
> with the next generation pass of course the complete site would be
> rendered... but this ran faster than the previous partial rebuild!
that really generates your menu doesn't use any information from the
node in which the menu is generated. Therefore the menu can be created
once and used on every page without going through the node_finder again.
However, the other statement, the one you have now commented out, use
node finder options that rely on information from the generated node
and therefore this menu has to be built for *each and every node*
separately. Meaning it needs to be built over 4000 times instead of
*once*.
What I'm trying to do now is to get more of the node finder options to
produce good cacheable results. This won't reduce the number to 1 but
it should be reducable to a few combinations of parameters (like node
level, node language) in most cases.
By the way: I saw that you manually create menus in many files, for
example in src/blog/pages/2010.page. By using a template (that
itself gets applied into the main default.template) that contains a
"{menu: ...}" you could automatically generate those.
Cheers,
Thomas
Lars Bähren
Feb 24, 2015, 4:28:07 AM2/24/15
to webgen...@googlegroups.com
On Tuesday, February 24, 2015 at 9:49:32 AM UTC+1, Thomas Leitner wrote:
Yeah, the reason why this works on your website is that the statement
that really generates your menu doesn't use any information from the
node in which the menu is generated. Therefore the menu can be created
once and used on every page without going through the node_finder again.
I switched back to the current version of the menu, because I was not able to get the more dynamic variant to work properly. Now that the overall generation time is considerably shorter I probably will have another go at that.
What I'm trying to do now is to get more of the node finder options to
produce good cacheable results. This won't reduce the number to 1 but
it should be reducable to a few combinations of parameters (like node
level, node language) in most cases.
Sounds great - I'll keep an eye on updates coming our way.
By the way: I saw that you manually create menus in many files, for
example in src/blog/pages/2010.page. By using a template (that
itself gets applied into the main default.template) that contains a
"{menu: ...}" you could automatically generate those.
I guess you are talking about the index pages. I am pretty aware that this is anything but an optimum solution - I guess some of this still is my lack of Ruby knowledge to figure a more Webgen-internal solution. Do you happen to have a pointer at some example code I could hae a look at to get a head start on this? I did some minor experimenting on 'archive.page', but the index pages requite a bit more work.
Cheers,
Lars
Thomas Leitner
Feb 24, 2015, 5:38:38 AM2/24/15
to webgen...@googlegroups.com
On 2015-02-24 01:28 -0800 Lars Bähren wrote:
> > By the way: I saw that you manually create menus in many files, for
> > example in src/blog/pages/2010.page. By using a template (that
> > itself gets applied into the main default.template) that contains a
> > "{menu: ...}" you could automatically generate those.
> >
>
> I guess you are talking about the index pages. I am pretty aware that
> this is anything but an optimum solution - I guess some of this still
> is my lack of Ruby knowledge to figure a more Webgen-internal
> solution. Do you happen to have a pointer at some example code I
> could hae a look at to get a head start on this? I did some minor
> experimenting on 'archive.page', but the index pages requite a bit
> more work.
webgen itself doesn't provide a blogging engine currently that
> > By the way: I saw that you manually create menus in many files, for
> > example in src/blog/pages/2010.page. By using a template (that
> > itself gets applied into the main default.template) that contains a
> > "{menu: ...}" you could automatically generate those.
> >
>
> I guess you are talking about the index pages. I am pretty aware that
> this is anything but an optimum solution - I guess some of this still
> is my lack of Ruby knowledge to figure a more Webgen-internal
> solution. Do you happen to have a pointer at some example code I
> could hae a look at to get a head start on this? I did some minor
> experimenting on 'archive.page', but the index pages requite a bit
> more work.
automatically generates archives, tag pages and other such pages.
For a fully automated solution for the index pages, you would need a
custom path handler extension that creates an index page for each
directory 2003, 2004, ... and 2004/01, 2004/02, ... and so on. See
http://webgen.gettalong.org/documentation/reference/api/Webgen/PathHandler.html#class-Webgen::PathHandler-label-Implementing+a+path+handler
for general implementation on how to implement one.
The content of the index pages could be completely generic - apart
from the list of links they are already very generic.
Each index page would need to point its 'template' meta information to
a special template that includes the logic for generating the links
(note that this can already be done and it is what I did to test the
template below; to be more specific, I added the "template" meta info
to 2014/index.page and 2014/01/index.page)
Here is a sample blog_index.template that should work for all the main
year and month directories (it still includes the index pages but this
can easily be fixed if necessary):
{menu: {options: {alcn: '<%= context.node.parent.alcn %>**/*.html'}}}
This generates a generic menu using list items and the title of the
blog entries. However, the menu tag allows you to use a custom template
(http://webgen.gettalong.org/documentation/reference/extensions/tag/menu.html#custom-menu-template)
so styling this or adding other information like the blog entry date
would be no problem.
Other things I found that could be optimized so that automating things
is easier:
* There is no blog entry date information available apart from in the
path of the file. This is okay but to automate things the entry
creation date would need to be stored in the 'created_at' meta info
(http://webgen.gettalong.org/documentation/reference/meta_information_keys.html#createdat).
This can be automated by a small extensions that adds the needed meta
information. See the source file lib/webgen/path_handler/meta_info.rb
for how this could be done.
* In the archives.page you access the file system. This should not be
because this might not always work correctly. Use the internal node
tree that webgen creates to access the nodes. It is available through
`context.node.tree`, or better still, use the node finder extension,
if possible, to get all the nodes you need. This allows integrating
your logic with webgen so that the archives page gets regenerated
automatically.
* You have .../blog/2014.page and .../blog/2014/index.page, however
only the latter is really necessary and from what I have seen they
have the same contents.
* The breadcrumb trail should probably use the omit_dir_index option
(see
http://webgen.gettalong.org/documentation/reference/configuration_options.html#tagbreadcrumbtrailomitdirindex).
Cheers,
Thomas
Thomas Leitner
Feb 24, 2015, 9:53:32 AM2/24/15
to webgen...@googlegroups.com
On 2015-02-23 23:28 -0800 Lars Bähren wrote:
> INFO [update] </writing/book_reviews/moondial.html>
> INFO [update] </writing/book_reviews/red_rabbit.html>
> INFO [update] </writing/book_reviews/the_divine_secrets_of_the_ya-ya-
> sisterhood.html>
> INFO [update]
> </writing/book_reviews/the_last_year_of_being_married.html> INFO
> [update] </writing/book_reviews/the_undutchables.html> INFO [update]
> </writing/index.html> INFO ... done in 256.40 seconds
> [100%] Built target Website
>
> I'll make a few local changes to get some numbers on the typical
> scenario of adding a new blog entry - but this already is impressive
> as is!
I have pushed another change that optimizes the use of the 'alcn',
> INFO [update] </writing/book_reviews/moondial.html>
> INFO [update] </writing/book_reviews/red_rabbit.html>
> INFO [update] </writing/book_reviews/the_divine_secrets_of_the_ya-ya-
> sisterhood.html>
> INFO [update]
> </writing/book_reviews/the_last_year_of_being_married.html> INFO
> [update] </writing/book_reviews/the_undutchables.html> INFO [update]
> </writing/index.html> INFO ... done in 256.40 seconds
> [100%] Built target Website
>
> I'll make a few local changes to get some numbers on the typical
> scenario of adding a new blog entry - but this already is impressive
> as is!
'absolute_levels', 'lang' and 'siblings' node finder options.
As I have written before these changes won't reduce the needed node
finder result sets to 1 but the improvements are nonetheless quite
impressiv.
When using webgen 1.3.0 it takes about 300s on my machine to generate
your original website (without my changes to the default.template).
With the next release it only takes about 130s!
So the commented out menu-tag that you had in your default.template
made webgen very slow but with the next release it will only be a
little bit slower.
Cheers,
Thomas
Reply all
Reply to author
Forward
0 new messages