Some improvements

[David Winslow]

Hey all,

I'll be presenting on Mapnik2GeoTools at FOSS4G this years, so I've spent some time cleaning things up and making this tool a bit more presentable.  The big new things are:

• mapnik2 compatibility
• a little UI to help you provide parameters (the pre-existing "batch" operation mode is still available)
• a binary distribution

You can grab the latest sources from Github, or download the pre-built version from https://github.com/downloads/dwins/mapnik2geotools/mn2gt-assembly-0.1.jar - as long as you have Java installed, you should be able to just double-click the JAR file in your file browser to run the new graphical interface.  Other options:

• run the GUI from a command-line: java -jar mn2gt-assembly-0.1.jar
• run the old batch interface from a command-line: java -cp mn2gt-assembly-0.1.jar mapquest-uk.xml --output output

It's definitely still a bit rough, but give it a try and let me know what you think :)

--
David Winslow
OpenGeo - http://opengeo.org/

[David Winslow]

Rough indeed! I got a bit ahead of myself with that last command-line.  The full thing should be:

java -cp mn2gt-assembly-0.1.jar me.winslow.d.mn2gt.Driver mapquest-uk.xml --output output

[fabian naumann]

Hey David,

i wanted to try your tool and downloaded first the german mapnik style
from the svn
(http://svn.openstreetmap.org/applications/rendering/mapnik-german/)
and your
mn2gt-assembly-0.1.jar. then i tried the comand line like you wrote:

java -cp /path/to/mn2gt-assembly-0.1.jar me.winslow.d.mn2gt.Driver /
path/to/mapnikstyle/osm-de.xml --output output

i get this error below which seems to be a syntax error in inc-de/
settings.xml.inc. i don´t know what i`m doing wrong.
i also tried your gui but the "coverting window" comes and nothing
happens. perhaps i forgot something.
I would be grateful if you could give me a tip. :-)

Fabian Naumann

Exception in thread "main" org.xml.sax.SAXParseException: The entity
name must immediately follow the '%' in the parameter entity
reference.
at
com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:
195)
at
com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:
174)
at
com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:
388)
at
com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:
1414)
at
com.sun.org.apache.xerces.internal.impl.XMLDTDScannerImpl.scanEntityValue(XMLDTDScannerImpl.java:
1686)
at
com.sun.org.apache.xerces.internal.impl.XMLDTDScannerImpl.scanEntityDecl(XMLDTDScannerImpl.java:
1571)
at
com.sun.org.apache.xerces.internal.impl.XMLDTDScannerImpl.scanDecls(XMLDTDScannerImpl.java:
1987)
at
com.sun.org.apache.xerces.internal.impl.XMLDTDScannerImpl.scanDTDInternalSubset(XMLDTDScannerImpl.java:
377)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl
$DTDDriver.dispatch(XMLDocumentScannerImpl.java:1141)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl
$DTDDriver.next(XMLDocumentScannerImpl.java:1090)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl
$PrologDriver.next(XMLDocumentScannerImpl.java:977)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:
648)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:
511)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:
808)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:
737)
at
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:
119)
at
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:
1205)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl
$JAXPSAXParser.parse(SAXParserImpl.java:522)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
at scala.xml.factory.XMLLoader$class.loadXML(XMLLoader.scala:
40)
at scala.xml.XML$.loadXML(XML.scala:40)
at scala.xml.factory.XMLLoader$class.load(XMLLoader.scala:54)
at scala.xml.XML$.load(XML.scala:40)
at me.winslow.d.mn2gt.driver.LocalConversion.run(driver.scala:
17)
at me.winslow.d.mn2gt.Driver$$anonfun$main
$1.apply(Driver.scala:59)
at me.winslow.d.mn2gt.Driver$$anonfun$main
$1.apply(Driver.scala:58)
at scala.collection.LinearSeqOptimized
$class.foreach(LinearSeqOptimized.scala:59)
at scala.collection.immutable.List.foreach(List.scala:45)
at me.winslow.d.mn2gt.Driver$.main(Driver.scala:58)
at me.winslow.d.mn2gt.Driver.main(Driver.scala)




On 2 Sep., 22:59, David Winslow <dwins...@opengeo.org> wrote:

> On Fri, Sep 2, 2011 at 4:55 PM, David Winslow <dwins...@opengeo.org> wrote:
> > Hey all,
>
> > I'll be presenting on Mapnik2GeoTools at FOSS4G this years, so I've spent
> > some time cleaning things up and making this tool a bit more presentable.
> > The big new things are:
>

> >    - mapnik2 compatibility
> >    - a little UI to help you provide parameters (the pre-existing "batch"

> >    operation mode is still available)

> >    - a binary distribution

>
> > You can grab the latest sources from Github, or download the pre-built
> > version from

> >https://github.com/downloads/dwins/mapnik2geotools/mn2gt-assembly-0.1...as long as you have Java installed, you should be able to just

> > double-click the JAR file in your file browser to run the new graphical
> > interface.  Other options:
>

> >    - run the GUI from a command-line: java -jar mn2gt-assembly-0.1.jar
> >    - run the old batch interface from a command-line: java -cp

> >    mn2gt-assembly-0.1.jar mapquest-uk.xml --output output
>
> > It's definitely still a bit rough, but give it a try and let me know what
> > you think :)
>
> > --
> > David Winslow

> > OpenGeo -http://opengeo.org/

[David Winslow]

The error reporting in the GUI is currently quite bad (on my todo-list: https://github.com/dwins/mapnik2geotools/issues/15 ), I recommend running via the command-line if things aren't working.  I also found some issues this weekend which I resolved but which are not fixed in the pre-built JAR file yet (one biggy: The "really publish to geoserver" option doesn't do anything at all!)  I'll try and get an updated build up today.

The first thing to check would be that you have substituted correct values for *all* of the placeholders (bits like "%(password)" ) in the *.inc.template files in the inc_de folder.

--
David Winslow
OpenGeo - http://opengeo.org/

[fabian naumann]

Thanks for the fast answer. okay that was a good tip. i substituted
all values but i don't understand why the fontset-settings.xml.inc is
necessary. do i have to install mapnik and the mapnik font dir? in the
fontset-settings.xml.inc is no placeholder, so i let everything. but
now comes:

Exception in thread "main" java.util.NoSuchElementException: key not
found: fontset-name

sorry i never worked with mapnik, only geoserver. so i was really
happy to find your tool. hope you can help me again.

Fabian Naumann


On 6 Sep., 19:21, David Winslow <dwins...@opengeo.org> wrote:
> The error reporting in the GUI is currently quite bad (on my todo-list:https://github.com/dwins/mapnik2geotools/issues/15), I recommend running

> via the command-line if things aren't working.  I also found some issues
> this weekend which I resolved but which are not fixed in the pre-built JAR
> file yet (one biggy: The "really publish to geoserver" option doesn't do
> anything at all!)  I'll try and get an updated build up today.
>
> The first thing to check would be that you have substituted correct values
> for *all* of the placeholders (bits like "%(password)" ) in the
> *.inc.template files in the inc_de folder.
>
> --
> David Winslow

> OpenGeo -http://opengeo.org/

> >https://github.com/downloads/dwins/mapnik2geotools/mn2gt-assembly-0.1...as you have Java installed, you should be able to just

[David Winslow]

Ok, looks like the problem you are running into is that the osm-de style is using Mapnik1.  When I implemented Mapnik2 support in Mapnik2GeoTools, I did so by replacing the existing code rather than reimplementing it all anew.  Mapnik2 changed a lot of XML element names to use - instead of _, ie, in the OSM-DE style it says "fontset_name" but in the MapQuest style (which is the one I usually use for testing) it says "fontset-name" instead.

If you're up for compiling your own copy of mn2gt it would be possible for you to go back a few versions and try with the one from before I updated to mapnik2 - if you're a little more industrious you could try updating the existing one to support both.  Otherwise, maybe you can try with the MapQuest styles instead - they are available from Github at https://github.com/MapQuest/MapQuest-Mapnik-Style

--
David Winslow
OpenGeo - http://opengeo.org/

[fabian naumann]

David thanks a lot! I didn´t know that the german style uses mapnik1.
i will try to compile the older version but updating to support both
is to hard for my actual knowledge in this theme. I tried for the
moment the mapquest style and it works! Thanks again. By the why. I
wish i could see your presentation on the foss4g but denver - germany
is a bit to far for me.

Fabian Naumann

On 6 Sep., 20:00, David Winslow <dwins...@opengeo.org> wrote:
> Ok, looks like the problem you are running into is that the osm-de style is
> using Mapnik1.  When I implemented Mapnik2 support in Mapnik2GeoTools, I did

> so by *replacing* the existing code rather than reimplementing it all anew.

> Mapnik2 changed a lot of XML element names to use - instead of _, ie, in the
> OSM-DE style it says "fontset_name" but in the MapQuest style (which is the
> one I usually use for testing) it says "fontset-name" instead.
>
> If you're up for compiling your own copy of mn2gt it would be possible for
> you to go back a few versions and try with the one from before I updated to
> mapnik2 - if you're a little more industrious you could try updating the
> existing one to support both.  Otherwise, maybe you can try with the

> MapQuest styles instead - they are available from Github athttps://github.com/MapQuest/MapQuest-Mapnik-Style
>
> --
> David Winslow
> OpenGeo -http://opengeo.org/

> >https://github.com/downloads/dwins/mapnik2geotools/mn2gt-assembly-0.1...have Java installed, you should be able to just

[David Winslow]

I'm creating a ticket to track this improvement idea: https://github.com/dwins/mapnik2geotools/issues/24

My slides from the FOSS4G talk will be available online after the talk.  I might even post them beforehand - but I haven't actually written them yet ;)

--
David Winslow
OpenGeo - http://opengeo.org/

[David Winslow]

I have updated the JAR file on Github with a newer version; this one enables automated GeoServer configuration and includes some postgres-related improvements from Nicolas Seinlet as well.

The URL is the same: https://github.com/downloads/dwins/mapnik2geotools/mn2gt-assembly-0.1.jar

--
David Winslow
OpenGeo - http://opengeo.org/

[Hailey Eckstrand]

Hi David,
I am trying to use your tool for styling OSM data that is using a
custom projection.
These are the steps that I have taken so far..
- transformed the geometry fields in postgis of the 4 planet_osm_*
tables to the projection I want (epsg:3005). The geometry fields are
still called 'way'.
- I have downloaded the mapquest mapnik styles from
https://github.com/MapQuest/MapQuest-Mapnik-Style
- downloaded the world_boundaries shapefiles & transformed them to
epsg:3005

- in the settings.xml.inc.. how do I specify a custom projection? This
is what mine is currently set to:
<!-- use the '&srs900913;' entity if you have called osm2pgsql without
special flags (or with -m); use '&srs4326;' if you have used -l -->
<!ENTITY osm2pgsql_projection "3005">

- in the datasource-settings.xml.inc I have set estimate extent to be
false & provided the extent in native coordinates (minx miny maxx
maxy)

Have I missed anything in pre-processing steps?
When I try to then run your tool, this is the error message I get:

$ java -cp mn2gt-assembly-0.1.jar me.winslow.d.mn2gt.Driver
mapquest_mapnik_style/mapquest-us.xml --output output
Exception in thread "main" java.io.FileNotFoundException:
mapquest_mapnik_style/output/world-raster-1.sld (No such file or
directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:194)
at java.io.FileOutputStream.<init>(FileOutputStream.java:145)
at java.io.FileWriter.<init>(FileWriter.java:73)
at me.winslow.d.mn2gt.driver.LocalConversion.save(driver.scala:
40)
at
me.winslow.d.mn2gt.driver.LocalConversion.writeStyle(driver.scala:65)
at me.winslow.d.mn2gt.driver.LocalConversion$$anonfun$run
$2.apply(driver.scala:32)
at me.winslow.d.mn2gt.driver.LocalConversion$$anonfun$run
$2.apply(driver.scala:31)
at scala.collection.Iterator$class.foreach(Iterator.scala:652)
at scala.collection.LinearSeqLike$$anon
$1.foreach(LinearSeqLike.scala:50)
at scala.collection.IterableLike
$class.foreach(IterableLike.scala:73)
at scala.xml.NodeSeq.foreach(NodeSeq.scala:43)
at me.winslow.d.mn2gt.driver.LocalConversion.run(driver.scala:
31)

at me.winslow.d.mn2gt.Driver$$anonfun$main
$1.apply(Driver.scala:59)
at me.winslow.d.mn2gt.Driver$$anonfun$main
$1.apply(Driver.scala:58)
at scala.collection.LinearSeqOptimized
$class.foreach(LinearSeqOptimized.scala:59)
at scala.collection.immutable.List.foreach(List.scala:45)
at me.winslow.d.mn2gt.Driver$.main(Driver.scala:58)
at me.winslow.d.mn2gt.Driver.main(Driver.scala)

Any help would be appreciated.
Cheers,
Hailey

[David Winslow]

Currently the converter doesn't actually support projections other than web mercator - but once the layers are set up you can reproject them on the fly just like any other layer in GeoServer.  In fact the data extent set by MN2GT is not dependent on the actual data involved at all.  So right now the path of least resistance is to import in epsg:900913 and request tiles in your custom projection, rather than converting all the data to your projection.

If someone with some coding skills wanted to make it work I could provide some pointers about where to look in the code...

Anyway, about your actual error - I suspect that you are trying to output to a directory that doesn't exist yet.  I made a couple of tweaks to the way that works (https://github.com/dwins/mapnik2geotools/commit/78434b9ebf6e8e811abc39e2d9f26abd4f81c998 shows the exact changes I made if you are interested).  I used to require that the output directory be created, now it will be created as long as the directory above it already exists, and the error messages when it doesn't work should be a bit nicer too.

I updated the downloads section, so you should be able to grab the jar file (it has the same name as the previous version, but I did update it) again and get the improvements.

Once things work, you'll get an "osm.sql" file along with the SLD output.  That script will create a number of pre-filtered tables with the OSM data for each layer, sort of custom-tailored to each style.  You'll need to run it (I use "psql -f osm.sql") before you try to auto-load the layers into GeoServer.

--

David Winslow

OpenGeo - http://opengeo.org/