locally translating/mapping URL addressing ...

[qwert...@syberianoutpost.ru]

~
I have been looking to no avail for some sort of RFC, standard implementation
or best practices for (a per-(sub)domain+URL-path flexible, local and functional
translations) of URL addressing. I think there might be such a thing since
spiders must use them in order to somehow organize and reference downloaded data,
including data contained by other data such as README text files and images in
tar balls.
~
In order to make local mappings more i18n-friendly and since FAT32/vfat is used
in many mobile devices and pen drives, based on RFC 3629; I would use UTF-8
character encoding tables for all non-ASCII and ASCII characters which aren't
in the 41 char sequence: "abcdefghijklmnopqrstuvwxyz0123456789-_.~"
~
Say you want to rewrite:
~
http://www.ted.com/talks/julian_assange_why_the_world_needs_wikileaks.html
http://download.ted.com/talks/JulianAssange_2010G-480p.mp4?apikey=TEDDOWNLOAD
~
as:
~
<local_dir>/com/ted/www/talks/julian_assange_why_the_world_needs_wikileaks.html
<local_dir>/com/ted/download/talks/2010G/JulianAssange-480p.mp4
~
and, of course, all URLs in the page must be translated to their local paths.
~
There must be some sort of in page xpath-like addressing, spidering/navigation
and page scraping as well; say, you only need the French and Arabic translations
of that talk. Also, you may need to selectively filter and/or consolidate the
content in pages as well, since there are many pages that present the same URL
with relative "previous", "next" links
~
The in-page, xpath-like content addressing can even be used for an
implementation of a deltas strategy at archive.org
~
Say you want to rewrite (java jar/zip files use "!" for addressing
compressed/contained resources):
~
http://www.gutenberg.org/files/24053/24053-h.zip/!/24053-h/images/illus-020l.jpg
~
as:
~
<local_dir>/org/gutenberg/www/files/24053/24053-h.zip/%21/24053-h/images/illus-020l.jpg
~
Say you want to download the main video data feed from a youtube page such as:
~
http://www.youtube.com/watch?v=n5pkDB7zEeo
~
as:
~
<local_dir>/com/youtube/www/TheOnion/n5pkDB7zEeo.mp4
~
including the youtube user, TheOnion, and the n5pkDB7zEeo part encoded
~
Say you want to download:
~
http://f1.grp.yahoofs.com/v1/OPsgUZByWMCDcqm6frjxW1ovKFsKIPhRQGjzpqK16ZETBxWjP
4gsRMF68Za5izYlJGnNgDRNOiSBkgcukXfrXajN5LKArQr1zA/
An%20Anti-Humean%20Principle%20for%20the%20Categories.pdf
~
as:
~
http://groups.yahoo.com/group/hegel/files/
An%20Anti-Humean%20Principle%20for%20the%20Categories.pdf
~
To read an english version of Cervantes' Don Quixote:
~
http://cervantes.thefreelibrary.com/Don-Quixote/6-1
~
they want for you to click, click, ... and click "next" (not even "previous")
~
... and many other cases which I am sure escape my mind
~
Any ideas/suggestions you would share?
~
thanks
lbrtchx
comp.infosystems.www.servers.unix:locally translating/mapping URL addressing ...