Download server broken? germany.map is only 21MB
314 views
Skip to first unread message
wish...@gmail.com
Jan 8, 2017, 5:43:54 AM1/8/17
to mapsforge-dev
Hey friends,
I was wondering whether there's some issue with the download server?
The current germany map is only 21mb, where I would expect a map size of 1gb+.
When opening the 21mb files in mapsforge, the map is widely empty, only city names are showing..
http://download.mapsforge.org/maps/europe/
Also, possibly unrelated I see a lot of old maps dating from early 2016, e.g.
Same issue when using the mirror at https://ftp-stud.hs-esslingen.de/pub/Mirrors/download.mapsforge.org/maps/
Any ideas?
Cheers
Toby
I was wondering whether there's some issue with the download server?
The current germany map is only 21mb, where I would expect a map size of 1gb+.
When opening the 21mb files in mapsforge, the map is widely empty, only city names are showing..
http://download.mapsforge.org/maps/europe/
| germany.map | 30-Dec-2016 14:47 | 21M |
Also, possibly unrelated I see a lot of old maps dating from early 2016, e.g.
|
Same issue when using the mirror at https://ftp-stud.hs-esslingen.de/pub/Mirrors/download.mapsforge.org/maps/
Any ideas?
Cheers
Toby
Emux
Jan 8, 2017, 7:19:50 AM1/8/17
to mapsfo...@googlegroups.com
Yes there was a problem with the build
process on the server.
It was reported and a fresh map creation cycle has already started.
It was reported and a fresh map creation cycle has already started.
--
Emux
Emux
Alan Taylor
Jan 13, 2017, 1:14:31 PM1/13/17
to mapsforge-dev
Any idea what's happening with this?
The GB files on this server haven't been updated since last April,
the German map is still only 21Mb.
The GB files on this server haven't been updated since last April,
the German map is still only 21Mb.
Tony Jay
Jan 15, 2017, 2:44:43 PM1/15/17
to mapsforge-dev
Hi,
How long does a build take as I see it still is 21MB
I have just ported my App to German and now I look like of a bit of an idiot as people downloading it for the first time think the App is broken.
Is there an archived version you can put in there (I have one if you don't !)
Best regards
Tony
Ludwig
Jan 15, 2017, 6:49:35 PM1/15/17
to mapsfo...@googlegroups.com
This might take some time as the whole build started from scratch.
Obviously I am not so happy that this has failed again, but at this occasion I also would like to remind people that we do not provide a backend server for your apps.
Ludwig
--
You received this message because you are subscribed to the Google Groups "mapsforge-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapsforge-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mapsfo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mapsforge-dev.
To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/bd8dfe20-a8ad-4416-b7b9-54751562a616%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Emux
Jan 16, 2017, 2:25:08 AM1/16/17
to mapsfo...@googlegroups.com
On 16/01/2017 01:49 πμ, Ludwig wrote:
I also would like to remind people that we do not provide a backend server for your apps.
Tony Jay
Jan 16, 2017, 3:26:31 AM1/16/17
to mapsforge-dev
Hi,
Thanks, does "some time" mean days, weeks of months (fibonacciestimate fine)
If the time is more than a few days, can you upload the previous version (I can send it if you do not have an archive ) ?
I don't use the server for my app (and I hope no one does as the advice is not to) , I use the mirror, and my user base is very low (50 new users a year )
regards
Tony
On Monday, January 16, 2017 at 12:49:35 AM UTC+1, Ludwig wrote:
This might take some time as the whole build started from scratch.Obviously I am not so happy that this has failed again, but at this occasion I also would like to remind people that we do not provide a backend server for your apps.Ludwig
On Jan 16, 2017 03:44, "Tony Jay" <anthony....@gmail.com> wrote:
Hi,--How long does a build take as I see it still is 21MBI have just ported my App to German and now I look like of a bit of an idiot as people downloading it for the first time think the App is broken.Is there an archived version you can put in there (I have one if you don't !)Best regardsTony
On Sunday, January 8, 2017 at 1:19:50 PM UTC+1, Emux wrote:Yes there was a problem with the build process on the server.
It was reported and a fresh map creation cycle has already started.
--
Emux
You received this message because you are subscribed to the Google Groups "mapsforge-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapsforge-de...@googlegroups.com.
Toby Karzel
Jan 19, 2017, 6:10:18 PM1/19/17
to mapsfo...@googlegroups.com
Any news on this? What's the status with the map build server?
In case you need support in a one-time effort to rebuild the maps, I might help recreating some of the defunct/outdated maps with my very basic map building stack and send them to you.
I consider the current situation as a critical issue: we mention the now defunct germany.map in all of our mapsforge examples, so any new user playing around with the example must get the wrong impression: mapsforge doesn't work.That's very bad and we should work together to solve the current map issue as soon as possible.
In case you need support in a one-time effort to rebuild the maps, I might help recreating some of the defunct/outdated maps with my very basic map building stack and send them to you.
I consider the current situation as a critical issue: we mention the now defunct germany.map in all of our mapsforge examples, so any new user playing around with the example must get the wrong impression: mapsforge doesn't work.That's very bad and we should work together to solve the current map issue as soon as possible.
--
You received this message because you are subscribed to a topic in the Google Groups "mapsforge-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mapsforge-dev/R0pd_ayX124/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mapsforge-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mapsfo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mapsforge-dev.
To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/56034d95-bc24-48d5-9f0c-9d65a114241e%40googlegroups.com.
Ludwig
Jan 21, 2017, 1:51:37 AM1/21/17
to mapsfo...@googlegroups.com
Just checked again and unfortunately Germany failed again, this is the error msg.
java.util.zip.DataFormatException: incorrect data check
at java.util.zip.Inflater.inflateBytes(Native Method)
at java.util.zip.Inflater.inflate(Inflater.java:259)
at java.util.zip.Inflater.inflate(Inflater.java:280)
at org.openstreetmap.osmosis.osmbinary.file.FileBlockPosition.parseData(FileBl
ockPosition.java:57)
at org.openstreetmap.osmosis.osmbinary.file.FileBlockHead.readContents(FileBlo
ckHead.java:95)
at org.openstreetmap.osmosis.osmbinary.file.FileBlock.process(FileBlock.java:1
35)
at org.openstreetmap.osmosis.osmbinary.file.BlockInputStream.process(BlockInpu
tStream.java:34)
at crosby.binary.osmosis.OsmosisReader.run(OsmosisReader.java:45)
at java.lang.Thread.run(Thread.java:745)
Jan 16, 2017 9:44:48 AM org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion
SCHWERWIEGEND: Thread for task 1-rb failed
java.lang.Error: java.util.zip.DataFormatException: incorrect data check
at org.openstreetmap.osmosis.osmbinary.file.FileBlockPosition.parseData(FileBlockPosition.java:60)
at org.openstreetmap.osmosis.osmbinary.file.FileBlockHead.readContents(FileBlockHead.java:95)
at org.openstreetmap.osmosis.osmbinary.file.FileBlock.process(FileBlock.java:135)
at org.openstreetmap.osmosis.osmbinary.file.BlockInputStream.process(BlockInputStream.java:34)
at crosby.binary.osmosis.OsmosisReader.run(OsmosisReader.java:45)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.zip.DataFormatException: incorrect data check
at java.util.zip.Inflater.inflateBytes(Native Method)
at java.util.zip.Inflater.inflate(Inflater.java:259)
at java.util.zip.Inflater.inflate(Inflater.java:280)
at org.openstreetmap.osmosis.osmbinary.file.FileBlockPosition.parseData(FileBlockPosition.java:57)
Basically there is an extraction error when getting the data for Germany. Since other countries work, this could have something to do with the size of the data extracted.
Anyone seen this kind of error before?
To unsubscribe from this group and stop receiving emails from it, send an email to mapsforge-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mapsfo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mapsforge-dev.
To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/CAGmvOhCpt8KhdGVBcmymDFR4gs_nZo-eWPkM%2Bs0%3DJTLUdWiMaA%40mail.gmail.com.
Tony Jay
Jan 21, 2017, 2:06:41 AM1/21/17
to mapsfo...@googlegroups.com
Can we please simply load up an archive version as an interim measure , I can give a dropbox link to my archive if someone knows how to upload to this server (hence the mirror will get updated)
Best regards
Tony
To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/CAAvpVPDVkZ%2BBOr92EPp2uphQHqSFEv-sJ3uBxxpL-u-sDrF_mg%40mail.gmail.com.
Ludwig
Jan 21, 2017, 2:27:43 AM1/21/17
to mapsfo...@googlegroups.com
Maybe we can make a deal: the problem is that the Germany file may be overwritten again unless we fix the scripts.
Emux already opened a ticket for only copying a new file in place if it is not substantially smaller.
The code for this copying is somewhere in https://github.com/mapsforge/mapsforge-mapcreator/blob/master/mapcreator.py
If someone puts the few lines of code in there that will do this basic verification, I will upload an old Germany file onto the server....
If someone sends me the link to the Germany file I can just download it.
Ludwig
To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/CAP0uTVmOz2yq_LoPgtq%3DnP1qehwC6frVp5z4eUFr3cjG5WD67w%40mail.gmail.com.
Emux
Jan 21, 2017, 8:46:55 AM1/21/17
to mapsfo...@googlegroups.com
Ludwig is right, or else we'll fall in
the same situation again..
BTW I have never seen such exception before.
My proposition is to change the map build process, the continents extraction seems to slow things considerably.
e.g. a major speedup could be to download .pbf from Geofabrik, merge with land / sea polygons and build directly with Osmosis (like described in map creation guide).
Of course that requires changes in the build scripts.. I opened issue #13 for that.
I created a new wiki in Mapsforge and VTM repositories with known Mapsforge map providers, for easy reference, please post any other free map provider you know.
BTW I have never seen such exception before.
My proposition is to change the map build process, the continents extraction seems to slow things considerably.
e.g. a major speedup could be to download .pbf from Geofabrik, merge with land / sea polygons and build directly with Osmosis (like described in map creation guide).
Of course that requires changes in the build scripts.. I opened issue #13 for that.
I created a new wiki in Mapsforge and VTM repositories with known Mapsforge map providers, for easy reference, please post any other free map provider you know.
--
Emux
Emux
wish...@gmail.com
Jan 21, 2017, 12:50:33 PM1/21/17
to mapsforge-dev
Sounds great!
I just added a proposal for the check at https://github.com/mapsforge/mapsforge-mapcreator/issues/12
I'm currently also building a new germany.map using osmosis from the latest geofabrik extracts.
As soon as it finishes, I'll post a download link..
Cheers
Toby
I just added a proposal for the check at https://github.com/mapsforge/mapsforge-mapcreator/issues/12
I'm currently also building a new germany.map using osmosis from the latest geofabrik extracts.
As soon as it finishes, I'll post a download link..
Cheers
Toby
Heinrich Göbl
Jan 21, 2017, 5:31:45 PM1/21/17
to mapsforge-dev
Well, restoring an old version of germany.map is great.
But you probably know, that most of the German "Bundesländer" (federal states?) maps are broken, too.
But you probably know, that most of the German "Bundesländer" (federal states?) maps are broken, too.
Would it be possible to change the create-script to proceed with other countries, even after an error occurred?
BTW I could supply an in Oct/2016 downloaded map of bayern (Bavaria).
Regarding the ZIP-error: Is the Zip-file still present on the server? What does "zip" command line tool say about the file? I suppose the ZIP-file is corrupt. Is there an MD5 hash available in the download source?
--
Heinrich
Heinrich Göbl
Jan 21, 2017, 5:40:17 PM1/21/17
to mapsforge-dev
Just found this: https://help.openstreetmap.org/questions/14206/incorrect-data-check
Maybe the temp space is too small?
Ludwig
Jan 22, 2017, 9:41:43 AM1/22/17
to mapsfo...@googlegroups.com
It might boil down to memory issues, I give the process 18GB of memory (out of 24 on the machine), but looking at the latest log for California this is not enough.
On 22 January 2017 at 06:40, Heinrich Göbl <heinric...@gmail.com> wrote:
Just found this: https://help.openstreetmap.org/questions/14206/incorrect-data-checkMaybe the temp space is too small?
--
You received this message because you are subscribed to the Google Groups "mapsforge-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapsforge-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mapsfo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mapsforge-dev.
To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/3a886f87-a5d7-43e0-8d87-5f4557c2e0d7%40googlegroups.com.
Heinrich Göbl
Jan 22, 2017, 11:15:25 AM1/22/17
to mapsforge-dev
If the machine is only running an Apache Http server, then you could probably give an maximum of 24GB.
Does the machine have a SSD harddisk? If yes, it should not be much of a problem to create swap on the SSD and even increase max-heap-size to more than physical RAM. But only as a last resort.
But I don't think that it's a matter of too small max-heap-size, because usually `OutOfMemoryError` occur in these cases.
And there are quite big map files created successfully (africa.map: 2.2GB).
Another thought: How much does the clipping cost when the .poly has hundreds of points? I just ask because for users like me living near the border to Austria and often crossing the border only a few kilometers, it's counterproductive to get the map clipped exactly at the political border. With little bike and hike-friendly "oversize" of about 5-10 kilometers I wouldn't have to change the map. I've found examples how to simplify and zoom the simpler shape so all points are included.
Do you think thats worth spending time? I could to that. I wouldn't change anything of the creation process, just provide simpler .poly files (of course with bigger areas).
--
Heinrich
Emux
Jan 22, 2017, 11:30:17 AM1/22/17
to mapsfo...@googlegroups.com
Border mapping is easily solved as
Mapsforge and VTM both support multiple maps rendering (see
screenshot), we have examples in both repositories.
It's a case already discussed many times in the forum and opted to proceed with defaults, as each user would suggest each time different offsets. :)
As I understand the exception is probably a bad download of planet osm from the start(?), which then fails at specific region because of missing binary data.
Instead of going back and forward I suggest to decide if we want to change the build process, my suggestion still stands.
I don't have currently time to change the scripts, but I'd like to read your thoughts on that.
It's a case already discussed many times in the forum and opted to proceed with defaults, as each user would suggest each time different offsets. :)
As I understand the exception is probably a bad download of planet osm from the start(?), which then fails at specific region because of missing binary data.
Instead of going back and forward I suggest to decide if we want to change the build process, my suggestion still stands.
I don't have currently time to change the scripts, but I'd like to read your thoughts on that.
--
Emux
Emux
Heinrich Göbl
Jan 22, 2017, 11:46:58 AM1/22/17
to mapsforge-dev
OK, multiple maps will solve my "user" problem. Thanks for pointing this out.
My intention in simplifying the .poly files was to improve map creation performance and save memory usage. If you think there's no correlation between problems in performance and the count of points in .poly files, then forget about it.
These are the countries with the most points:
3533 ./europe/germany/bayern/oberbayern.poly
3799 ./russia/far-eastern-federal-district/magadanskaya.poly
3830 ./russia/far-eastern-federal-district/kamchatsky.poly
3867 ./russia/southern-federal-district.poly
3956 ./russia/northwestern-federal-district/komi.poly
4769 ./russia/siberian-federal-district/zabaykalsky.poly
4913 ./russia/ural-federal-district/khanty-mansiysky.poly
4971 ./russia/siberian-federal-district/buryatiya.poly
5076 ./russia/far-eastern-federal-district/amurskaya.poly
5166 ./russia/ural-federal-district/yamalo-nenetsky.poly
5556 ./russia/central-federal-district.poly
6721 ./russia/far-eastern-federal-district/khabarovsky.poly
7648 ./russia/siberian-federal-district/irkutskaya.poly
7660 ./russia/volga-federal-district.poly
8793 ./russia/northwestern-federal-district.poly
8874 ./russia/ural-federal-district.poly
9118 ./russia/siberian-federal-district/krasnoyarsky.poly
9817 ./europe/france/alsace/bas-rhin.poly
10067 ./russia/far-eastern-federal-district.poly
10165 ./russia/far-eastern-federal-district/sakha-yakutiya.poly
14557 ./russia/siberian-federal-district.poly
18751 ./europe/france/aquitaine/gironde.poly
I'd really like to help you. I'm afraid your suggestion of solving this problem is too complex for me at this moment.
@Emux, how much time would you estimate to finish this task?
Emux
Jan 22, 2017, 12:00:12 PM1/22/17
to mapsfo...@googlegroups.com
And how about abandon the .poly files
and produce maps filling the whole rectangle bounds of the view?
That way we can have map data around borders (which is a good suggestion) and also handle only simple rectangles, like we do already with land and sea.
Though in that case we'd need a planet osm as input.
That way we can have map data around borders (which is a good suggestion) and also handle only simple rectangles, like we do already with land and sea.
Though in that case we'd need a planet osm as input.
--
Emux
Emux
Andrew
Jan 26, 2017, 4:54:18 AM1/26/17
to mapsforge-dev
Any update on restoring the old maps for Germany until the new ones are generated correctly?
Anyone can provide links to the last version of the old maps?
Thanks!
Anyone can provide links to the last version of the old maps?
Thanks!
Andrew
Jan 26, 2017, 5:46:19 AM1/26/17
to mapsforge-dev
If you need some working maps, I found this old mirror from Dec 2014: http://download.fumuga.com/mapsforge-maps
Better than nothing...
Better than nothing...
Ludwig
Jan 26, 2017, 9:38:02 AM1/26/17
to mapsfo...@googlegroups.com
The server has started working on Europe, with Germany coming first. Let's see what happens....
On 26 January 2017 at 18:46, Andrew <ajmo...@gmail.com> wrote:
If you need some working maps, I found this old mirror from Dec 2014: http://download.fumuga.com/mapsforge-maps
Better than nothing...
--
You received this message because you are subscribed to the Google Groups "mapsforge-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapsforge-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mapsfo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mapsforge-dev.
To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/246d8410-9f29-4dff-9710-802d6201b40a%40googlegroups.com.
Ludwig
Jan 26, 2017, 11:21:41 PM1/26/17
to mapsfo...@googlegroups.com
Of course an error in the new script, but I restarted it and it correctly skipped the process of rebuilding recently built pbf files, the server is currently extracting the pbf for Germany....
Emux
Jan 27, 2017, 2:26:18 AM1/27/17
to mapsfo...@googlegroups.com
On 27/01/2017 06:21 πμ, Ludwig wrote:
Of course an error in the new script, but I restarted it and it correctly skipped the process of rebuilding recently built pbf files
Heinrich Göbl
Jan 30, 2017, 4:30:06 AM1/30/17
to mapsforge-dev
Seams like it's working now.
2017/.../germany is doing well. Congratulations! Please don't overwrite at least until it's in the mirror.
Thanks a lot for your efforts.
--
Heinrich
Am Freitag, 27. Januar 2017 05:21:41 UTC+1 schrieb Ludwig:
Of course an error in the new script, but I restarted it and it correctly skipped the process of rebuilding recently built pbf files, the server is currently extracting the pbf for Germany....
On 26 January 2017 at 22:38, Ludwig <ludwigbr...@gmail.com> wrote:
The server has started working on Europe, with Germany coming first. Let's see what happens....
On 26 January 2017 at 18:46, Andrew <ajmo...@gmail.com> wrote:
If you need some working maps, I found this old mirror from Dec 2014: http://download.fumuga.com/mapsforge-maps
Better than nothing...
--
You received this message because you are subscribed to the Google Groups "mapsforge-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapsforge-de...@googlegroups.com.
To post to this group, send email to mapsfo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mapsforge-dev.
Alan Taylor
Jan 30, 2017, 4:40:50 AM1/30/17
to mapsforge-dev
What about Great Britain?
It still hasn't been updated since April last year.
It still hasn't been updated since April last year.
Ludwig
Jan 30, 2017, 7:36:21 AM1/30/17
to mapsfo...@googlegroups.com
The server is currently making the Bundesländer in Germany.
Then comes France, then Britain, then other countries in Europe.
Ludwig
On 30 January 2017 at 17:40, 'Alan Taylor' via mapsforge-dev <mapsfo...@googlegroups.com> wrote:
What about Great Britain?
It still hasn't been updated since April last year.
--
You received this message because you are subscribed to the Google Groups "mapsforge-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapsforge-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mapsfo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mapsforge-dev.
To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/9340aaef-9eeb-4534-98ce-f44b730a59b2%40googlegroups.com.
Alan Taylor
Jan 31, 2017, 2:26:05 PM1/31/17
to mapsforge-dev
It take it the build process died again?
Ludwig
Jan 31, 2017, 5:15:05 PM1/31/17
to mapsfo...@googlegroups.com
No, but the land of Nordrhein-Wesfalen takes a long time, it is Germany's largest state....
On 1 February 2017 at 03:26, 'Alan Taylor' via mapsforge-dev <mapsfo...@googlegroups.com> wrote:
It take it the build process died again?
--
You received this message because you are subscribed to the Google Groups "mapsforge-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapsforge-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mapsfo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mapsforge-dev.
To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/3f4c8a50-69f5-455b-9736-f8d678f83508%40googlegroups.com.
Ludwig
Feb 1, 2017, 9:15:32 AM2/1/17
to mapsfo...@googlegroups.com
I killed the job for Germany Nordrhein-Westfalen, it ran out of memory and then got stuck. If we are lucky and the scripts work, maybe the next run will be more successful.
On 1 February 2017 at 06:15, Ludwig <ludwigbr...@gmail.com> wrote:
No, but the land of Nordrhein-Wesfalen takes a long time, it is Germany's largest state....
Emux
Feb 1, 2017, 9:20:28 AM2/1/17
to mapsfo...@googlegroups.com
The whole country was built right, but
a region failed?
Can have a new process for each map file build, so that VM is initialized each time?
Can have a new process for each map file build, so that VM is initialized each time?
--
Emux
Emux
Heinrich Göbl
Feb 1, 2017, 9:24:44 AM2/1/17
to mapsforge-dev
Would it be possible to send a signal to the Java process so that
a stack-trace is dumped to console? (Next time it starves...)
This would give some insight to what's going wrong. I cannot
believe it's really running out of memory, because there would be
a OutOfMemoryError. Maybe paging of memory (thrashing) is taking
too much time. This is as well a signal for too less memory, but
theoretically the process could end successfully. Can you monitor
page-faults, CPU, Thread-count, ...?
Ludwig
Feb 2, 2017, 9:47:12 PM2/2/17
to mapsfo...@googlegroups.com
I restarted the process because of another error in the script, it correctly skipped all the recently generated maps, but failed again on that large state in Germany, nordrhein-westfalen with OOM. This time though it did not hang, but the process was terminated, and the map creation process is currently working on the france country map.
This was the OOM (just for reference, yes it is strange that germany.map works, but a subset not):
Feb 02, 2017 8:15:24 AM org.openstreetmap.osmosis.core.Osmosis runINFORMATION: Osmosis Version 0.43.1
Feb 02, 2017 8:15:24 AM org.openstreetmap.osmosis.core.Osmosis run
INFORMATION: Preparing pipeline.
Feb 02, 2017 8:15:25 AM org.mapsforge.map.writer.osmosis.MapFileWriterTask <init>
INFORMATION: mapfile-writer version: mapsforge-map-writer-master-SNAPSHOT
Feb 02, 2017 8:15:25 AM org.mapsforge.map.writer.osmosis.MapFileWriterTask <init>
INFORMATION: mapfile format specification version: 3
Feb 02, 2017 8:15:25 AM org.openstreetmap.osmosis.core.Osmosis run
INFORMATION: Launching pipeline execution.
Feb 02, 2017 8:15:25 AM org.openstreetmap.osmosis.core.Osmosis run
INFORMATION: Pipeline executing, waiting for completion.
Feb 02, 2017 8:39:42 AM org.mapsforge.map.writer.osmosis.MapFileWriterTask complete
INFORMATION: completing read...
Feb 02, 2017 8:43:28 AM org.mapsforge.map.writer.osmosis.MapFileWriterTask complete
INFORMATION: start writing file...
Feb 02, 2017 8:43:31 AM org.mapsforge.map.writer.util.PolyLabel get
WARNUNG: Failed to get label for geometry: MULTILINESTRING ((6.495692 50.575446, 6.495863 50.575369, 6.
495966 50.575462, 6.495795 50.575539, 6.495809 50.575555, 6.495656 50.575623, 6.495284 50.575288, 6.495
324 50.575272, 6.495243 50.575195, 6.495109 50.575254, 6.494998 50.575154, 6.495259 50.575036, 6.49537
50.575137, 6.495326 50.575158, 6.495347 50.575181, 6.49537 50.57517, 6.495407 50.575203, 6.495384 50.57
5214, 6.495406 50.575235, 6.495437 50.57522, 6.495692 50.575446))
Feb 02, 2017 8:43:31 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 100% of sub file for zoom interval 1 (5,0,7)
Feb 02, 2017 8:43:31 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 10% of sub file for zoom interval 2 (10,8,11)
Feb 02, 2017 8:43:33 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 20% of sub file for zoom interval 2 (10,8,11)
Feb 02, 2017 8:43:38 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 30% of sub file for zoom interval 2 (10,8,11)
Feb 02, 2017 8:43:42 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 40% of sub file for zoom interval 2 (10,8,11)
Feb 02, 2017 8:43:44 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 50% of sub file for zoom interval 2 (10,8,11)
Feb 02, 2017 8:43:48 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 60% of sub file for zoom interval 2 (10,8,11)
Feb 02, 2017 8:43:51 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 70% of sub file for zoom interval 2 (10,8,11)
Feb 02, 2017 8:43:51 AM org.mapsforge.map.writer.util.PolyLabel get
WARNUNG: Failed to get label for geometry: MULTILINESTRING ((6.495692 50.575446, 6.495863 50.575369, 6.
495966 50.575462, 6.495795 50.575539, 6.495809 50.575555, 6.495656 50.575623, 6.495284 50.575288, 6.495
324 50.575272, 6.495243 50.575195, 6.495109 50.575254, 6.494998 50.575154, 6.495259 50.575036, 6.49537
50.575137, 6.495326 50.575158, 6.495347 50.575181, 6.49537 50.57517, 6.495407 50.575203, 6.495384 50.57
5214, 6.495406 50.575235, 6.495437 50.57522, 6.495692 50.575446))
Feb 02, 2017 8:43:52 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 80% of sub file for zoom interval 2 (10,8,11)
Feb 02, 2017 8:43:53 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 90% of sub file for zoom interval 2 (10,8,11)
Feb 02, 2017 8:43:55 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 10% of sub file for zoom interval 3 (14,12,21)
Feb 02, 2017 11:28:52 AM org.mapsforge.map.writer.MapFileWriter processTile
WARNUNG: error in parallel preprocessing of ways
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.mapsforge.map.writer.MapFileWriter.processTile(MapFileWriter.java:816)
at org.mapsforge.map.writer.MapFileWriter.writeSubfile(MapFileWriter.java:906)
at org.mapsforge.map.writer.MapFileWriter.writeFile(MapFileWriter.java:361)
at org.mapsforge.map.writer.osmosis.MapFileWriterTask.complete(MapFileWriterTask.java:102)
at org.openstreetmap.osmosis.set.v0_6.EntityMerger.run(EntityMerger.java:243)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.mapsforge.map.writer.util.PolyLabel.get(PolyLabel.java:123)
at org.mapsforge.map.writer.MapFileWriter$WayPreprocessingCallable.call(MapFileWriter.java:193)
at org.mapsforge.map.writer.MapFileWriter$WayPreprocessingCallable.call(MapFileWriter.java:94)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
Feb 02, 2017 11:29:05 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 20% of sub file for zoom interval 3 (14,12,21)
Feb 02, 2017 11:29:30 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 30% of sub file for zoom interval 3 (14,12,21)
Feb 02, 2017 12:09:28 PM org.mapsforge.map.writer.MapFileWriter processTile
WARNUNG: error in parallel preprocessing of ways
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.mapsforge.map.writer.MapFileWriter.processTile(MapFileWriter.java:816)
at org.mapsforge.map.writer.MapFileWriter.writeSubfile(MapFileWriter.java:906)
at org.mapsforge.map.writer.MapFileWriter.writeFile(MapFileWriter.java:361)
at org.mapsforge.map.writer.osmosis.MapFileWriterTask.complete(MapFileWriterTask.java:102)
at org.openstreetmap.osmosis.set.v0_6.EntityMerger.run(EntityMerger.java:243)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.mapsforge.map.writer.util.PolyLabel.get(PolyLabel.java:123)
at org.mapsforge.map.writer.MapFileWriter$WayPreprocessingCallable.call(MapFileWriter.java:193)
at org.mapsforge.map.writer.MapFileWriter$WayPreprocessingCallable.call(MapFileWriter.java:94)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
Feb 02, 2017 12:09:42 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry prior to tile clipping, trying to repair 378371611
Feb 02, 2017 12:09:42 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry even after attempt to fix 378371611
Feb 02, 2017 12:09:44 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry prior to tile clipping, trying to repair 379840815
Feb 02, 2017 12:09:44 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry even after attempt to fix 379840815
Feb 02, 2017 12:09:44 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry prior to tile clipping, trying to repair 295291230
Feb 02, 2017 12:09:44 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry even after attempt to fix 295291230
Feb 02, 2017 12:09:45 PM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 40% of sub file for zoom interval 3 (14,12,21)
Feb 02, 2017 12:09:57 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry prior to tile clipping, trying to repair 159250148
Feb 02, 2017 12:09:57 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry even after attempt to fix 159250148
Feb 02, 2017 1:10:33 PM org.mapsforge.map.writer.MapFileWriter processTile
WARNUNG: error in parallel preprocessing of ways
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.mapsforge.map.writer.MapFileWriter.processTile(MapFileWriter.java:816)
at org.mapsforge.map.writer.MapFileWriter.writeSubfile(MapFileWriter.java:906)
at org.mapsforge.map.writer.MapFileWriter.writeFile(MapFileWriter.java:361)
at org.mapsforge.map.writer.osmosis.MapFileWriterTask.complete(MapFileWriterTask.java:102)
at org.openstreetmap.osmosis.set.v0_6.EntityMerger.run(EntityMerger.java:243)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.mapsforge.map.writer.util.PolyLabel.get(PolyLabel.java:122)
at org.mapsforge.map.writer.MapFileWriter$WayPreprocessingCallable.call(MapFileWriter.java:193)
at org.mapsforge.map.writer.MapFileWriter$WayPreprocessingCallable.call(MapFileWriter.java:94)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
Feb 02, 2017 1:11:00 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry prior to tile clipping, trying to repair 262414510
Feb 02, 2017 1:11:04 PM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 50% of sub file for zoom interval 3 (14,12,21)
Feb 02, 2017 1:15:36 PM org.mapsforge.map.writer.MapFileWriter processTile
WARNUNG: error in parallel preprocessing of ways
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.mapsforge.map.writer.MapFileWriter.processTile(MapFileWriter.java:816)
at org.mapsforge.map.writer.MapFileWriter.writeSubfile(MapFileWriter.java:906)
at org.mapsforge.map.writer.MapFileWriter.writeFile(MapFileWriter.java:361)
at org.mapsforge.map.writer.osmosis.MapFileWriterTask.complete(MapFileWriterTask.java:102)
at org.openstreetmap.osmosis.set.v0_6.EntityMerger.run(EntityMerger.java:243)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3181)
at java.util.PriorityQueue.grow(PriorityQueue.java:300)
at java.util.PriorityQueue.offer(PriorityQueue.java:339)
at java.util.PriorityQueue.add(PriorityQueue.java:321)
at org.mapsforge.map.writer.util.PolyLabel.get(PolyLabel.java:124)
at org.mapsforge.map.writer.MapFileWriter$WayPreprocessingCallable.call(MapFileWriter.java:193)
at org.mapsforge.map.writer.MapFileWriter$WayPreprocessingCallable.call(MapFileWriter.java:94)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
Feb 02, 2017 1:15:45 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry prior to tile clipping, trying to repair 161285424
Feb 02, 2017 1:15:45 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry even after attempt to fix 161285424
Feb 02, 2017 1:15:46 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry prior to tile clipping, trying to repair 262414148
Feb 02, 2017 1:15:54 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry prior to tile clipping, trying to repair 399069878
Feb 02, 2017 1:15:54 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry even after attempt to fix 399069878
Feb 02, 2017 1:16:27 PM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 60% of sub file for zoom interval 3 (14,12,21)
Feb 02, 2017 1:16:29 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry prior to tile clipping, trying to repair 401606518
Feb 02, 2017 1:16:29 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry even after attempt to fix 401606518
Feb 02, 2017 3:47:17 PM org.mapsforge.map.writer.MapFileWriter processTile
WARNUNG: error in parallel preprocessing of ways
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.mapsforge.map.writer.MapFileWriter.processTile(MapFileWriter.java:816)
at org.mapsforge.map.writer.MapFileWriter.writeSubfile(MapFileWriter.java:906)
at org.mapsforge.map.writer.MapFileWriter.writeFile(MapFileWriter.java:361)
at org.mapsforge.map.writer.osmosis.MapFileWriterTask.complete(MapFileWriterTask.java:102)
at org.openstreetmap.osmosis.set.v0_6.EntityMerger.run(EntityMerger.java:243)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
Feb 02, 2017 4:09:50 PM org.mapsforge.map.writer.MapFileWriter processTile
WARNUNG: error in parallel preprocessing of ways
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.mapsforge.map.writer.MapFileWriter.processTile(MapFileWriter.java:816)
at org.mapsforge.map.writer.MapFileWriter.writeSubfile(MapFileWriter.java:906)
at org.mapsforge.map.writer.MapFileWriter.writeFile(MapFileWriter.java:361)
at org.mapsforge.map.writer.osmosis.MapFileWriterTask.complete(MapFileWriterTask.java:102)
at org.openstreetmap.osmosis.set.v0_6.EntityMerger.run(EntityMerger.java:243)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
Feb 02, 2017 4:09:50 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry prior to tile clipping, trying to repair 35031957
Feb 02, 2017 4:09:50 PM org.mapsforge.map.writer.util.GeoUtils clipToTile
WARNUNG: invalid geometry even after attempt to fix 35031957
Feb 02, 2017 4:13:45 PM org.mapsforge.map.writer.MapFileWriter processTile
WARNUNG: error in parallel preprocessing of ways
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.mapsforge.map.writer.MapFileWriter.processTile(MapFileWriter.java:816)
at org.mapsforge.map.writer.MapFileWriter.writeSubfile(MapFileWriter.java:906)
at org.mapsforge.map.writer.MapFileWriter.writeFile(MapFileWriter.java:361)
at org.mapsforge.map.writer.osmosis.MapFileWriterTask.complete(MapFileWriterTask.java:102)
at org.openstreetmap.osmosis.set.v0_6.EntityMerger.run(EntityMerger.java:243)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
Feb 02, 2017 4:15:51 PM org.mapsforge.map.writer.MapFileWriter processTile
WARNUNG: error in parallel preprocessing of ways
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.mapsforge.map.writer.MapFileWriter.processTile(MapFileWriter.java:816)
at org.mapsforge.map.writer.MapFileWriter.writeSubfile(MapFileWriter.java:906)
at org.mapsforge.map.writer.MapFileWriter.writeFile(MapFileWriter.java:361)
at org.mapsforge.map.writer.osmosis.MapFileWriterTask.complete(MapFileWriterTask.java:102)
at org.openstreetmap.osmosis.set.v0_6.EntityMerger.run(EntityMerger.java:243)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
--
You received this message because you are subscribed to the Google Groups "mapsforge-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapsforge-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mapsfo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mapsforge-dev.
To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/c2721aad-3f07-4579-9167-64a0c68e2b63%40googlegroups.com.
Heinrich Göbl
Feb 4, 2017, 10:13:40 AM2/4/17
to mapsforge-dev
Some of the OOM are "Java heap space", others are "GC overhead limit exceeded".
Regarding the latter, have a look at http://stackoverflow.com/a/1393503/2176962 and http://stackoverflow.com/a/5839392/2176962
Is there a chance for some kind of memory leak? Which objects are held in memory while the map is created/written?
Is there a way to run the "Nordrhein-Westfalen" with exactly the same intermediate files (created pbf or the like)?
Then one could monitor the process with jconsole and make a heap-dump to see how many objects (HashMap, String, ...) fly around the universe.
There is some "parallel preprocessing". Is it possible that there are really many parallel threads doing some work? Is there an executor reducing the amount of parallel threads? If so, could the number of threads be reduced (just to save memory)?
What could be the reason that "germany.map" succeeds, while the smaller "nordrhein-westfalen" fails? I've seen that the polygons for smaller countries are cut finer than the surrounding countries. What else differs?
@Ludwig, can you provide all the input files and the command line so we can try to reproduce the behaviour?
Longri, Andre Höpfner
Feb 4, 2017, 10:25:37 AM2/4/17
to mapsfo...@googlegroups.com
I think that the biggest problem is Osmosis.
In the attempts I made last year, it turned out that Osmosis unpacked the complete * .pbf and kept it in RAM. This behavior I find not good, but I have given up for not enough of time to write my own map-writer.
--
You received this message because you are subscribed to the Google Groups "mapsforge-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapsforge-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mapsfo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mapsforge-dev.
To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/361f6b4a-a7a5-4a38-8478-3a3361cd4e56%40googlegroups.com.
Ludwig
Feb 8, 2017, 12:56:40 AM2/8/17
to mapsfo...@googlegroups.com
The process failed on the Netherlands again (with OOM), so I have made some quick fixes to reduce the number of internal threads when writing the mapfile, see
https://github.com/mapsforge/mapsforge/issues/920, and restarted the process on the server.To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/CAMHyUuPkNngQHS%3DVmh_8J_YmMbiiLh1%2B%3DdZvjCMOWWsXdF0qMA%40mail.gmail.com.
Ludwig
Feb 16, 2017, 9:25:56 PM2/16/17
to mapsfo...@googlegroups.com
A couple of failures again, with OOM.
The latvia process was hanging for three days, until I killed it off.
I have now installed the latest mapwriter with the changes that Emux made (functionally equivalent to my changes) and have removed the intermediate pbf files for Europe as a whole and for Germany. In Germany the NordrheinWestfalen map failed again, maybe building from planet directly will improve this. This might also hold for Italy, the Netherlands etc.
I restarted the process, just trying to make everything that is older than 60 days, this should go a bit quicker.
Ludwig
Emux
Feb 17, 2017, 2:57:32 AM2/17/17
to mapsfo...@googlegroups.com
On 17/02/2017 04:25 πμ, Ludwig wrote:
have removed the intermediate pbf files...
...maybe building from planet directly will improve this.
That's a good plan.
BTW in my local builds I use the --read-pbf-fast with workers the cores number.
Another recommended improvement is the --buffer with a value e.g. 10000.
--
Emux
Emux
Ludwig
Feb 21, 2017, 7:29:47 PM2/21/17
to mapsfo...@googlegroups.com
The server once again hung on making the latvia map, I think it got stuck at the same point last time:
INFORMATION: written 60% of sub file for zoom interval 2 (10,8,11)
Feb 19, 2017 2:58:36 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 70% of sub file for zoom interval 2 (10,8,11)
Feb 19, 2017 2:58:38 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 80% of sub file for zoom interval 2 (10,8,11)
Feb 19, 2017 2:58:39 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 90% of sub file for zoom interval 2 (10,8,11)
Feb 19, 2017 2:58:41 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 10% of sub file for zoom interval 3 (14,12,21)
Feb 19, 2017 2:58:55 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 20% of sub file for zoom interval 3 (14,12,21)
Feb 19, 2017 2:59:46 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 30% of sub file for zoom interval 3 (14,12,21)
Feb 19, 2017 3:00:52 AM org.mapsforge.map.writer.MapFileWriter writeSubfile
INFORMATION: written 40% of sub file for zoom interval 3 (14,12,21)
Feb 19, 2017 3:01:16 AM org.mapsforge.map.writer.MapFileWriter$WayPreprocessingCallable call
INFORMATION: Large geometry 32951623696 (2483 coords, down from 3362 coords)
The other European countries still missing have failed again with OOM errors, currently working on the Netherlands.
--
You received this message because you are subscribed to the Google Groups "mapsforge-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapsforge-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mapsfo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mapsforge-dev.
To view this discussion on the web visit https://groups.google.com/d/msgid/mapsforge-dev/f2cf80e1-55d0-d523-2250-851fb5aa36c9%40gmail.com.
Ludwig
Feb 27, 2017, 5:19:20 AM2/27/17
to mapsfo...@googlegroups.com
I have changed the remaining areas to use the HD processor, this seems to have solved the problem and now we have new maps for france, italy etc.
We do have intermittent GC problems, I am not so happy with that as I do not clearly understand if that will maybe cause problems with the maps integrity.
If I understand this correctly this GC overhead exception is a performance indicator, so it is not actually a failure in the file, but it leads to a missing element in the map. There are some ways to turn this off, at the risk of programs just doing GC and nothing else.
I will enable this on the next run which I will start once the current process is finished.
But otherwise we are finally back to a relatively new set of maps for all areas.
Thanks for your patience and for pointing out the problems in the first place....
Ludwig
Reply all
Reply to author
Forward
0 new messages