All Russian map.

[Антон Лунегов]

Hello! I'm trying create russian map with Osmosis. I have 1.5Gb Russia.osm.pbf file. But i can't create it. Please, see, what i'm doing wrong? P.s: computers configuration - SSD and 14 Gb RAM for Java.

C:\maps\osmosis-latest\bin>osmosis.bat --rb file=C:\maps\RU.osm.pbf --mapfile-wr

iter file=C:\maps\Ru.map type=hd

ряЁ 15, 2015 2:03:52 PM org.openstreetmap.osmosis.core.Osmosis run

INFO: Osmosis Version 0.43.1

ряЁ 15, 2015 2:03:52 PM org.openstreetmap.osmosis.core.Osmosis run

INFO: Preparing pipeline.

ряЁ 15, 2015 2:03:52 PM org.mapsforge.map.writer.osmosis.MapFileWriterTask <init

>

INFO: mapfile-writer version: mapsforge-map-writer-0.5.1

ряЁ 15, 2015 2:03:52 PM org.mapsforge.map.writer.osmosis.MapFileWriterTask <init

>

INFO: mapfile format specification version: 3

ряЁ 15, 2015 2:03:52 PM org.openstreetmap.osmosis.core.Osmosis run

INFO: Launching pipeline execution.

ряЁ 15, 2015 2:03:52 PM org.openstreetmap.osmosis.core.Osmosis run

INFO: Pipeline executing, waiting for completion.

ряЁ 15, 2015 2:03:53 PM org.mapsforge.map.writer.osmosis.MapFileWriterTask proce

ss

INFO: start reading data...

ряЁ 15, 2015 2:09:56 PM org.mapsforge.map.writer.osmosis.MapFileWriterTask compl

ete

INFO: completing read...

ряЁ 15, 2015 3:39:18 PM org.mapsforge.map.writer.osmosis.MapFileWriterTask compl

ete

INFO: start writing file...

ряЁ 15, 2015 3:39:19 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 10.0% of sub file for zoom interval index 0

ряЁ 15, 2015 3:39:21 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 20.0% of sub file for zoom interval index 0

ряЁ 15, 2015 3:39:22 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 30.0% of sub file for zoom interval index 0

ряЁ 15, 2015 3:39:27 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 40.0% of sub file for zoom interval index 0

ряЁ 15, 2015 3:39:37 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 50.0% of sub file for zoom interval index 0

ряЁ 15, 2015 3:39:44 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 60.0% of sub file for zoom interval index 0

ряЁ 15, 2015 3:39:56 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 70.0% of sub file for zoom interval index 0

ряЁ 15, 2015 3:40:17 PM org.mapsforge.map.writer.util.GeoUtils clipToTile

WARNING: invalid geometry prior to tile clipping, trying to repair 331691805

ряЁ 15, 2015 3:40:17 PM org.mapsforge.map.writer.util.GeoUtils clipToTile

WARNING: invalid geometry even after attempt to fix 331691805

ряЁ 15, 2015 3:40:37 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 80.0% of sub file for zoom interval index 0

ряЁ 15, 2015 3:41:07 PM org.mapsforge.map.writer.util.GeoUtils clipToTile

WARNING: invalid geometry prior to tile clipping, trying to repair 305495807

ряЁ 15, 2015 3:41:07 PM org.mapsforge.map.writer.util.GeoUtils clipToTile

WARNING: invalid geometry even after attempt to fix 305495807

ряЁ 15, 2015 3:41:17 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 90.0% of sub file for zoom interval index 0

ряЁ 15, 2015 3:41:33 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 100% of sub file for zoom interval index 0

ряЁ 15, 2015 3:41:39 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 10.0% of sub file for zoom interval index 1

ряЁ 15, 2015 3:41:50 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 20.0% of sub file for zoom interval index 1

ряЁ 15, 2015 3:41:57 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 30.0% of sub file for zoom interval index 1

ряЁ 15, 2015 3:42:25 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 40.0% of sub file for zoom interval index 1

ряЁ 15, 2015 3:43:42 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 50.0% of sub file for zoom interval index 1

ряЁ 15, 2015 3:46:22 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336152524 (2212 coords, down from 72501 coords)

ряЁ 15, 2015 3:46:23 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336152524 (3011 coords, down from 72501 coords)

ряЁ 15, 2015 3:46:29 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336152524 (4225 coords, down from 72501 coords)

ряЁ 15, 2015 3:46:38 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336151457 (4098 coords, down from 36681 coords)

ряЁ 15, 2015 3:46:44 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336151855 (3692 coords, down from 25055 coords)

ряЁ 15, 2015 3:46:59 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 60.0% of sub file for zoom interval index 1

ряЁ 15, 2015 3:47:16 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336225085 (3003 coords, down from 18301 coords)

ряЁ 15, 2015 3:47:16 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336225196 (2013 coords, down from 15830 coords)

ряЁ 15, 2015 3:47:21 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336224671 (2808 coords, down from 18045 coords)

ряЁ 15, 2015 3:47:22 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336224826 (2597 coords, down from 11450 coords)

ряЁ 15, 2015 3:47:40 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336287794 (3451 coords, down from 18324 coords)

ряЁ 15, 2015 3:47:41 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336287796 (3627 coords, down from 18644 coords)

ряЁ 15, 2015 3:47:49 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336287793 (2698 coords, down from 13550 coords)

ряЁ 15, 2015 3:47:52 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336287792 (2202 coords, down from 11519 coords)

ряЁ 15, 2015 3:47:52 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336287798 (2239 coords, down from 20447 coords)

ряЁ 15, 2015 3:48:01 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336287795 (2752 coords, down from 18280 coords)

ряЁ 15, 2015 3:48:03 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336287797 (2323 coords, down from 19440 coords)

ряЁ 15, 2015 3:48:05 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336153619 (3054 coords, down from 21905 coords)

ряЁ 15, 2015 3:48:05 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336287798 (3107 coords, down from 20447 coords)

ряЁ 15, 2015 3:48:06 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336153620 (2204 coords, down from 47512 coords)

ряЁ 15, 2015 3:48:06 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336153620 (2851 coords, down from 47512 coords)

ряЁ 15, 2015 3:48:15 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336287797 (3818 coords, down from 19440 coords)

ряЁ 15, 2015 3:48:16 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336287799 (3827 coords, down from 21961 coords)

ряЁ 15, 2015 3:48:19 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336153620 (3074 coords, down from 47512 coords)

ряЁ 15, 2015 3:48:19 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336153620 (3244 coords, down from 47512 coords)

ряЁ 15, 2015 3:48:20 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336153620 (2184 coords, down from 47512 coords)

ряЁ 15, 2015 3:48:38 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336287799 (2356 coords, down from 21961 coords)

ряЁ 15, 2015 3:51:00 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336226988 (3665 coords, down from 22029 coords)

ряЁ 15, 2015 3:52:35 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336227992 (3251 coords, down from 9051 coords)

ряЁ 15, 2015 3:52:35 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336227987 (2061 coords, down from 5040 coords)

ряЁ 15, 2015 3:52:57 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336227995 (3372 coords, down from 9293 coords)

ряЁ 15, 2015 3:52:58 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336227988 (2303 coords, down from 4972 coords)

ряЁ 15, 2015 3:53:19 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336228845 (2535 coords, down from 14987 coords)

ряЁ 15, 2015 3:53:19 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336228847 (3114 coords, down from 20894 coords)

ряЁ 15, 2015 3:53:20 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336225619 (2724 coords, down from 25856 coords)

ряЁ 15, 2015 3:53:20 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336225619 (2513 coords, down from 25856 coords)

ряЁ 15, 2015 3:53:58 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 70.0% of sub file for zoom interval index 1

ряЁ 15, 2015 3:54:08 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336152785 (3348 coords, down from 16739 coords)

ряЁ 15, 2015 3:55:55 PM org.mapsforge.map.writer.util.GeoUtils clipToTile

WARNING: invalid geometry prior to tile clipping, trying to repair 169303351

ряЁ 15, 2015 3:55:55 PM org.mapsforge.map.writer.util.GeoUtils clipToTile

WARNING: invalid geometry even after attempt to fix 169303351

ряЁ 15, 2015 3:56:26 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 293768247 (2268 coords, down from 13283 coords)

ряЁ 15, 2015 4:01:06 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 108097335 (2501 coords, down from 6835 coords)

ряЁ 15, 2015 4:02:23 PM org.mapsforge.map.writer.util.GeoUtils clipToTile

WARNING: invalid geometry prior to tile clipping, trying to repair 331691805

ряЁ 15, 2015 4:02:23 PM org.mapsforge.map.writer.util.GeoUtils clipToTile

WARNING: invalid geometry even after attempt to fix 331691805

ряЁ 15, 2015 4:06:49 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336257477 (3351 coords, down from 19886 coords)

ряЁ 15, 2015 4:07:09 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336209258 (2555 coords, down from 6444 coords)

ряЁ 15, 2015 4:07:09 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336238144 (2330 coords, down from 6482 coords)

ряЁ 15, 2015 4:08:57 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336153018 (2595 coords, down from 8155 coords)

ряЁ 15, 2015 4:12:55 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 80.0% of sub file for zoom interval index 1

ряЁ 15, 2015 4:18:19 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336152614 (2835 coords, down from 8572 coords)

ряЁ 15, 2015 4:20:46 PM org.mapsforge.map.writer.util.GeoUtils clipToTile

WARNING: invalid geometry prior to tile clipping, trying to repair 305495807

ряЁ 15, 2015 4:20:46 PM org.mapsforge.map.writer.util.GeoUtils clipToTile

WARNING: invalid geometry even after attempt to fix 305495807

ряЁ 15, 2015 4:25:16 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 90.0% of sub file for zoom interval index 1

ряЁ 15, 2015 4:27:29 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336238394 (2584 coords, down from 20011 coords)

ряЁ 15, 2015 4:27:43 PM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingC

allable call

INFO: Large geometry 336242105 (2700 coords, down from 43142 coords)

ряЁ 15, 2015 4:30:31 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 10.0% of sub file for zoom interval index 2

ряЁ 15, 2015 4:35:41 PM org.mapsforge.map.writer.MapFileWriter writeSubfile

INFO: written 20.0% of sub file for zoom interval index 2

ряЁ 15, 2015 7:56:18 PM org.openstreetmap.osmosis.core.store.RandomAccessObjectS

tore release

WARNING: Unable to delete file C:\Users\Alexandr\AppData\Local\Temp\idxNodesd475

134546693778521.tmp

ряЁ 15, 2015 7:56:18 PM org.openstreetmap.osmosis.core.store.RandomAccessObjectS

tore release

WARNING: Unable to delete file C:\Users\Alexandr\AppData\Local\Temp\idxNodesi764

5447232969849705.tmp

ряЁ 15, 2015 7:56:18 PM org.openstreetmap.osmosis.core.store.RandomAccessObjectS

tore release

WARNING: Unable to delete file C:\Users\Alexandr\AppData\Local\Temp\idxWaysd4008

271601876560985.tmp

ряЁ 15, 2015 7:56:18 PM org.openstreetmap.osmosis.core.store.RandomAccessObjectS

tore release

WARNING: Unable to delete file C:\Users\Alexandr\AppData\Local\Temp\idxWaysi5640

328506158963720.tmp

ряЁ 15, 2015 7:56:18 PM org.openstreetmap.osmosis.core.pipeline.common.ActiveTas

kManager waitForCompletion

SEVERE: Thread for task 1-rb failed

java.lang.OutOfMemoryError: GC overhead limit exceeded

        at gnu.trove.list.array.TLongArrayList.<init>(TLongArrayList.java:91)

        at gnu.trove.list.array.TLongArrayList.<init>(TLongArrayList.java:67)

        at org.mapsforge.map.writer.HDTileData.<init>(HDTileData.java:33)

        at org.mapsforge.map.writer.HDTileBasedDataProcessor.getTileImpl(HDTileB

asedDataProcessor.java:255)

        at org.mapsforge.map.writer.HDTileBasedDataProcessor.getTile(HDTileBased

DataProcessor.java:213)

        at org.mapsforge.map.writer.MapFileWriter.processTile(MapFileWriter.java

:760)

        at org.mapsforge.map.writer.MapFileWriter.writeSubfile(MapFileWriter.jav

a:906)

        at org.mapsforge.map.writer.MapFileWriter.writeFile(MapFileWriter.java:3

63)

        at org.mapsforge.map.writer.osmosis.MapFileWriterTask.complete(MapFileWr

iterTask.java:99)

        at crosby.binary.osmosis.OsmosisBinaryParser.complete(OsmosisBinaryParse

r.java:35)

        at org.openstreetmap.osmosis.osmbinary.file.BlockInputStream.process(Blo

ckInputStream.java:37)

        at crosby.binary.osmosis.OsmosisReader.run(OsmosisReader.java:45)

        at java.lang.Thread.run(Unknown Source)

ряЁ 15, 2015 7:56:18 PM org.openstreetmap.osmosis.core.Osmosis main

SEVERE: Execution aborted.

org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks failed

.

        at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForComple

tion(Pipeline.java:146)

        at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:92)

        at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

        at java.lang.reflect.Method.invoke(Unknown Source)

        at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Laun

cher.java:329)

        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.jav

a:239)

        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(La

uncher.java:409)

        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:

352)

        at org.codehaus.classworlds.Launcher.main(Launcher.java:47)

[fzk]

Look like insufficient memory:

java.lang.OutOfMemoryError: GC overhead limit exceeded

Klaus

[Emux]

Yes memory issue (how much physical RAM?).

Make sure you're playing with 64bit Java and you can try also diffetent zoom-interval-conf.

--
Emux
Cruiser - Atlas

[Антон Лунегов]

16 Gb physical RAM. yep, 64-bit Java. How i can add different-zoom-interval-conf? Thank you.

понедельник, 20 апреля 2015 г., 13:03:26 UTC+5 пользователь Emux написал:

[Emux]

Check the instructions for a low detail world map implementation here,
which uses a different zoom-interval-conf from the default.

[Антон Лунегов]

Ok. How about "type=hd" - parameter? It's nessesary?

понедельник, 20 апреля 2015 г., 13:40:46 UTC+5 пользователь Emux написал:

[Emux]

If you're going to play with such a huge region then it's needed (or you'll need a lot of RAM).

[Антон Лунегов]

Thank you. I will try)

понедельник, 20 апреля 2015 г., 13:49:29 UTC+5 пользователь Emux написал:

[Антон Лунегов]

Hello again. I was trying creating all Russia map with this parameters: C:\maps\osmosis-latest\bin>osmosis.bat --rbf file=C:\maps\RU.osm.pbf workers=4 --mapfile-writer file=C:\maps\Ru.map type=ram zoom-interval-conf=7,0,15 type=hd .But it was not working.

When i set zoom-interval-conf max-parameter = 14, it's work! But map has not had houses. Help me please. 

понедельник, 20 апреля 2015 г., 13:49:29 UTC+5 пользователь Emux написал:

If you're going to play with such a huge region then it's needed (or you'll need a lot of RAM).

[Emux]

At your command line there is 2 times the "type" parameter.

Another thing to try is to allow Osmosis pipeline to be split across multiple threads with e.g.:
--b bufferCapacity=10000

Also you can modify your Java options (depending your JRE version), e.g.:
JAVACMD_OPTIONS="-XX:+AggressiveOpts -XX:+UseCompressedOops -Djava.util.Arrays.useLegacyMergeSort=true -XX:PermSize=256m -XX:MaxPermSize=256m -Xms8g -Xmx8g -server"

Zoom interval conf defines what zoom level groups will be in the map file. If you stop it at 14, then zoom 15 elements will not appear.

World map has one low zoom level interval because it does not need to show detailed data.

[Антон Лунегов]

Thank you very much! I did it, when i set bufferCapacity=50000.

среда, 27 мая 2015 г., 12:44:13 UTC+5 пользователь Emux написал:

[Emux]

On 28/05/2015 07:30 πμ, Антон Лунегов wrote:

Thank you very much! I did it, when i set bufferCapacity=50000.

You were able to build the whole Russia map with zoom levels down to 20+ ?

Could you provide the Osmosis command line and Java options you used, many would be interested for that.

[Антон Лунегов]

No, i was usinf 15+  zoom levels down. And it's fine for me. But now I'm having all Russia map without Moscow))

четверг, 28 мая 2015 г., 11:09:20 UTC+5 пользователь Emux написал:

[Emux]

You can make specific elements appear earlier if you use a custom tag-mapping and render theme.