[TW5] New plugin for generating sitemap.xml for static sites

[Jim Lehmer]

I cloned off my ATOM feed plugin and created one for generating a sitemap.xml file. I have verified the results on my web site successfully against Google Webmaster tools. You can see the resulting file here.

I have submitted it to Google as well as updated my robots.txt file to point to it (I have my robots.txt file in my TW and it gets output as part of my "generate site" command). For reference, my robots.txt looks like this.

You can see more about the plugin on GitHub. It's still rough around the edges, but it works for my purposes. Note that it prioritizes the home page (a tiddler named Home or index, case-insensitive) and the site index (a tiddler named SiteIndex, case-insensitive) above other "normal" pages.

Comments welcome.

[Tobias Beer]

Cool stuff, Jim.

With the increasing information on static pages,

I have started to collect some notes on the different tidbits...

http://static.tiddlyspot.com

Best wishes, Tobias.

[Jim Lehmer]

On Monday, January 5, 2015 9:31:01 AM UTC-6, Tobias Beer wrote:

Cool stuff, Jim.

With the increasing information on static pages,

I have started to collect some notes on the different tidbits...

http://static.tiddlyspot.com

Thanks, Tobias!

[sini-Kit]

it don't work on my http://heeg.ru/heeg.html when I open sitemap tiddler I get a red mistake  "substring"  NULL
 version of tw is 5.1.5
понедельник, 5 января 2015 г., 1:05:55 UTC+3 пользователь Jim Lehmer написал:

[Jim Lehmer]

On 1/18/2015 5:51 AM, sini-Kit wrote:

it don't work on my http://heeg.ru/heeg.html when I open sitemap tiddler I get a red mistake  "substring"  NULL
 version of tw is 5.1.5

Interesting.

It looks like your site is a "live" Javascript-enabled TW, not a statically-generated set of pages. I didn't really have that use case in mind when I built the plugin, but it should still work.

The only place in my plugin that uses substrings is in a date formatting function in sitemapentries.js, taking TW's internal date format, such as:

20141123155239707

...and converting it to the format sitemap wants:

2014-11-23

(Sitemap doesn't care about the time portion, and Google's verifier will throw a warning if it is included.)

The function just appends three substrings of the TW date string, plus two hyphens:

XML.twDateToWebDate = function(twDate) {
	return twDate.substring(0, 4) + "-" + twDate.substring(4, 6) + "-" + twDate.substring(6, 8);
}

So, if the error is happening in my code, that tells me there is a tiddler or tiddlers without a modified field. I would have always expected that field to get created (it seems to on any tiddlers I create). I wonder if some system tiddlers are built without the modified field? Does anyone else know?

When you use the plugin, are you passing it a filter to filter out system tiddlers?

[Jim Lehmer]

On 1/18/2015 7:40 AM, Jim Lehmer wrote:

So, if the error is happening in my code, that tells me there is a tiddler or tiddlers without a modified field. I would have always expected that field to get created (it seems to on any tiddlers I create). I wonder if some system tiddlers are built without the modified field? Does anyone else know?

And the answer is, "Yes," there are lots of system tiddlers in the node version (and I would expect in the standalone version) that don't have either created or modified fields.

When you use the plugin, are you passing it a filter to filter out system tiddlers?

Which then leads to needing to filter out system tiddlers (which should be filtered out, anyway, because they're not going to be meaningful in a sitemap).

[Tobias Beer]

So, what to do with tiddlers that don't have a modified date and a sitemap?

Is it ok for them not having any?

I guess you want to use a custom filter anyway,

if only to not index system tiddlers

...or just all tagged "blog" or whatever you use

to distinguish content from overhead.

Best wishes, Tobias.

[Jim Lehmer]

That was precisely my case - I can't see any reason to not filter out system tiddlers.

I note in the sitemap spec that the lastmod element is optional, so I will probably end up changing the code to only call the date formatting routine and generate that element if there is actually a modified field, just to insulate from this error. Will go create an issue on Github to remind myself of it now.

[PMario]

On Sunday, January 18, 2015 at 2:40:36 PM UTC+1, Jim Lehmer wrote:

The function just appends three substrings of the TW date string, plus two hyphens:

XML.twDateToWebDate = function(twDate) {
	return twDate.substring(0, 4) + "-" + twDate.substring(4, 6) + "-" + twDate.substring(6, 8);
}

The TW date string is still subject for discussion. So if you expect the format to be like this, your plugin may break in the future.
There is a $tw.utils.formatDateString() function that can do the conversion.

also see: http://tiddlywiki.com/#DateFormat
 

[PMario]

see:  core/modules/utlis/utils.js

exports.formatDateString = function(date,template)

-----

$tw.utils.formatDateString(new Date, "YYYY-MMM")
will give you "2015-January"

-m

[Jim Lehmer]

Good to know. I will log another issue in my repo. :)

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/Xton0F9QtHE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.

[sini-Kit]

Hi Jim! I saw your last modification, it is better, but not work with Russian letters :(

if you want, you can make sitemap without any macros

look my site here http://novye-podarki.ru/heeg.html#%24%3A%2F_sitemap

only one tiddler......

понедельник, 19 января 2015 г., 0:04:09 UTC+3 пользователь Jim Lehmer написал: