Fwd: [OSM-talk] Archive.org Tile Serving

[Jeffrey Warren]

whoa... i wonder if Archive.org would host a copy of the PLOTS Archive's map tiles/geotiffs...?

---------- Forwarded message ----------
From: Stefan de Konink <ste...@konink.de>
Date: Wed, Apr 25, 2012 at 7:18 AM
Subject: [OSM-talk] Archive.org Tile Serving
To: ta...@openstreetmap.org, d...@openstreetmap.org


Hi,

Last night Archive.org has set up a collection to facilitate tile serving from Archive.org. We can experiment on what the best structure of this would be. Anyone with interest in this subject please contact me.


Stefan

_______________________________________________
talk mailing list
ta...@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk

[Jeffrey Warren]

interesting conversation on that topic at OpenStreetMap here: http://wiki.openstreetmap.org/wiki/TileCDN#Archive.org.

It'd be really nice to have all our maps archived externally! 

[Liz Barry]

This would be great, a real support. We'd have to assess our archive, here's an excerpt of what archive.org wrote to OSM:

Since items on archive.org are best kept under 10GB total and less than 1000 total files per item it would be good to have a mapping scheme that makes the best use. Less Items with more files is better for us so maybe you do it by latlon or region.

-- Post to this group at publicla...@googlegroups.com. To unsubscribe, email publiclaborato...@googlegroups.com. Options at https://groups.google.com/d/forum/publiclaboratory?hl=en

--
@lizbarry

[Jeffrey Warren]

i think this is because OSM only wants to host map tiles... we could request the same even if they wouldn't be able to host the GeoTiffs.

[Adam Griffith]

+1

[Alex Mandel]

FYI, Amazon has a plan to host public data on S3 for free too, cost is
when you want to transfer it out. Also I can discuss with OpenArealMap
and OSGeo if you'd like them to try and mirror your data.

Could you give me an estimate of current space and growth per month?

Thanks,
Alex

On 04/25/2012 07:02 AM, Jeffrey Warren wrote:
> i think this is because OSM only wants to host map tiles... we could
> request the same even if they wouldn't be able to host the GeoTiffs.
>
> On Wed, Apr 25, 2012 at 9:50 AM, Liz Barry <eba...@gmail.com> wrote:
>
>> This would be great, a real support. We'd have to assess our archive,
>> here's an excerpt of what archive.org wrote to OSM:
>>

>> *Since items on archive.org are best kept under 10GB total and less than

>> 1000 total files per item it would be good to have a mapping scheme that
>> makes the best use. Less Items with more files is better for us so maybe

>> you do it by latlon or region.*

>>
>> On Wed, Apr 25, 2012 at 9:32 AM, Jeffrey Warren <je...@unterbahn.com>wrote:
>>
>>> interesting conversation on that topic at OpenStreetMap here:
>>> http://wiki.openstreetmap.org/wiki/TileCDN#Archive.org.
>>>
>>> It'd be really nice to have all our maps archived externally!
>>>
>>>
>>> On Wed, Apr 25, 2012 at 9:28 AM, Jeffrey Warren <je...@unterbahn.com>wrote:
>>>
>>>> whoa... i wonder if Archive.org would host a copy of the PLOTS Archive's
>>>> map tiles/geotiffs...?
>>>>
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: Stefan de Konink <ste...@konink.de>
>>>> Date: Wed, Apr 25, 2012 at 7:18 AM
>>>> Subject: [OSM-talk] Archive.org Tile Serving
>>>> To: ta...@openstreetmap.org, d...@openstreetmap.org
>>>>
>>>>
>>>> Hi,
>>>>
>>>> Last night Archive.org has set up a collection to facilitate tile
>>>> serving from Archive.org. We can experiment on what the best structure of
>>>> this would be. Anyone with interest in this subject please contact me.
>>>>
>>>>
>>>> Stefan
>>>>

>>>> ______________________________**_________________
>>>> talk mailing list
>>>> ta...@openstreetmap.org
>>>> http://lists.openstreetmap.**org/listinfo/talk<http://lists.openstreetmap.org/listinfo/talk>

>>>>
>>>>
>>> -- Post to this group at publicla...@googlegroups.com. To
>>> unsubscribe, email publiclaborato...@googlegroups.com.
>>> Options at https://groups.google.com/d/forum/publiclaboratory?hl=en
>>>
>>
>>
>>
>> --

>> @lizbarry <http://twitter.com/lizbarry>

[Jeffrey Warren]

Awesome! The archive is about a terabyte right now, with geotiffs ranging from a few hundred mb to a couple gb each, and is growing at a rate of about 1/2 TB per year though we can expect this to accelerate.

Stewart, this sounds right, no?

Jeff

[Alex Mandel]

telascience might be able to mirror that. Is it all in a tilecache?
If so I've been told that all we need to give telascience is the url to
the tilecache.cfg and they can make a mirror of it.

Thanks,
Alex

[Jeffrey Warren]

It's all in various formats, you can browse at archive.publiclaboratory.org

we have both flat files in TMS-formatted directories AND raw geotiffs. Either one would be helpful to archive, though Geotiffs probably are better for long-term storage while TMS directories are better for actually serving to people. 

Jeff