DNC Zoom in problem

[Ryan Jones]

I am not sure if this is possible given varying levels of resolution in data for different geographical areas but when I zoom in, some areas can be zoomed in all the way while others give an exception and the attached image is the result. Here is the exception

Exception in thread "DncRenderCache4" java.lang.NullPointerException

at com.metsci.glimpse.dnc.convert.Flat2Render$TransitionalDatabase.createChunk(Flat2Render.java:894)

at com.metsci.glimpse.dnc.convert.Flat2Render$RenderCache.convertChunk(Flat2Render.java:567)

at com.metsci.glimpse.dnc.convert.Flat2Render$RenderCache$1.runThrows(Flat2Render.java:498)

at com.metsci.glimpse.dnc.convert.Flat2Render$DncChunkJob.run(Flat2Render.java:1130)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Any help would be great.

Thanks,

Ryan

[Mike Hogye]

Progress!

There are two issues here: (1) zooming in results in blank space, and (2) the NullPointerException.

(1)

When you zoom in far enough, you will get a level-of-detail that doesn't cover the whole world (e.g. the "harbor" LOD only has data right around harbors). Right now, in that situation, we just show blank space where there's no data. It's not great behavior -- would be better to do something like fall back to data from a coarser LOD -- and it's on our list to fix.

The LOD behavior is customizable. When you do "new DncPainter()", you pass in a DncPainterSettings object. By default this is an instance of DncPainterSettingsImpl, which has the naive behavior. You could write a subclass of DncPainterSettingsImpl, and override its isLibraryActive() method to do something more intelligent.

You could, for example, have the "general" LOD always be visible, in addition to finer LODs as appropriate. (LODs are drawn from coarsest to finest, so if you have a "general" library and a "harbor" library that overlap, the "harbor" library will appear in front.)

There are probably nicer ways to do it than that, involving (among other things) checkbox widgets elsewhere in your GUI. We hope to get there eventually.

(2)

The NPE is strange. I can give you a direction to investigate, but I'm not sure exactly what the problem is.

Based on the stack dump, it looks like it's trying to load a coverage that doesn't exist in the database. Where you call "dncPainter.activateCoverages()", try activating fewer coverages -- maybe just "coa" and "ecr" -- and hopefully the NPE will go away. Then, add the other coverage names back in gradually, and see which of them triggers the NPE.

Then, see if the problematic coverage name is missing from your "DNC_FLAT/dncflat13/coverage-names" file (which is just a text file).

Let me know what you find!

[Mike Hogye]

To be clear: I don't think these two issues are related to each other.

It's possible I'm wrong about that, especially if the NPE doesn't happen until you hit a particular zoom level. Let me know what you find, in any case.

[Ryan Jones]

Overriding the isLibraryActive() worked well. I removed all the layers except the ecr and coa and still am getting exceptions. my coverage-names file has all of the coverages. I also removed ecr and it still gave me errors. I will look more closely at it. It definitely happens while zooming to certain levels though. It seems to work just fine despite those exceptions though.

[Mike Hogye]

Can you attach the text you get on stderr, up through at least one NPE? There's a logging.properties file in glimpse-extras-examples/dnc-examples/ you can use.

At the beginning you should get a bunch of lines like this:

  FINER: main - Flat2Render - Found externally converted chunk: database = 13, library = GEN13A, coverage = nav

And later, when it needs a chunk that isn't in the cache yet, lines like this:

  FINER: DncRenderCache2 - Flat2Render - Converting chunk: late-priority = IMMEDIATE, early-priority = IMMEDIATE, deferrals = 0, total-wait = 425 ms, database = 13, library = GEN13A, coverage = rel

It might be instructive to look at which chunk it was trying to convert right before the NPE, on the thread on which the NPE occurred.

[Mike Hogye]

If your IDE can go into debug mode when an exception is thrown, you could run until you hit the NPE, and then poke around a little in the debugger. At Flat2Render.java:894, the things to inspect are:

• this.flatDir
• this.flatCoverageNums
• coverageName

It must be the case that this.flatCoverageNums (which is a map from String to int) does not contain an entry for the coverageName in question. The map gets loaded from the "coverage-names" file inside the directory pointed to by this.flatDir -- so if coverageName is missing from the map, it's worth confirming that it's also missing from the file.

I think we're pretty careful about uppercase vs. lowercase coverage names ... but it could conceivably be something along those lines. Maybe extra whitespace at the end of a line? If you can attach the "coverage-names" file for the problematic database, I can stare at it for a while and look for garbage like that.

[Ryan Jones]

Sorry I haven't had a chance to get back to you. Unfortunately I had to put this aside for a while because currently I am restricted to needing to support Java 1.7 for my tools and the DNC portion requires java 1.8 since it uses lambda expressions. I'll let you know if I jump back into using it.

-Ryan

[Mike Hogye]

How long do you expect to be on Java 1.7? The glimpse team has had a running discussion about whether/when to move to 1.8; would be helpful to hear an outside perspective.

Changing the DNC code to work on Java 1.7 is certainly within the realm of possibility.

[Ryan Jones]

I  am not sure how long we will be on java 1.7. It is completely out of my hands unfortunately. I imagine it we will switch at some point soon but that may be wishful thinking.

-Ryan

[Mike Hogye]

I found the NPE issue -- DncPainterExample is trying to activate a coverage (or possibly more than one) that doesn't exist in the publicly available DNC data.

In DncPainterExample, find the "activateCoverages" call, and replace it with:

        dncPainter.activateCoverages( renderCache.coverages );

That should be it.

[Ryan Jones]

I just now have gotten back to trying to work on this. I tried starting fresh using DNC13 only. I have all of the files for this downloaded. I ran your vpf2flatconverter program. Left the flat directory to the default location. Ran the program and get this error.

 CONFIG: AWT-EventQueue-0 - GlimpseDataPaths - Glimpse shared data: C:\ProgramData\Glimpse\Data

 CONFIG: AWT-EventQueue-0 - GlimpseDataPaths - Glimpse user data: C:\Users\rjones\AppData\Local\Glimpse\Data

 CONFIG: AWT-EventQueue-0 - GlimpseDataPaths - Glimpse user cache: C:\Users\rjones\AppData\Local\Glimpse\Cache

   INFO: AWT-EventQueue-0 - DncDataPaths - Default DNC_FLAT user dir: C:\Users\rjones\AppData\Local\Glimpse\Data\DNC_FLAT

   INFO: AWT-EventQueue-0 - DncDataPaths - Default DNC_FLAT shared dir: C:\ProgramData\Glimpse\Data\DNC_FLAT

   INFO: AWT-EventQueue-0 - DncDataPaths - Standard DNC_FLAT dir: C:\Users\rjones\AppData\Local\Glimpse\Data\DNC_FLAT

   INFO: AWT-EventQueue-0 - DncDataPaths - Standard DNC_RENDER dir: C:\Users\rjones\AppData\Local\Glimpse\Cache\DNC_RENDER

   INFO: AWT-EventQueue-0 - DncDataPaths - Standard DNC_QUERY dir: C:\Users\rjones\AppData\Local\Glimpse\Cache\DNC_QUERY

Exception in thread "Converter" java.lang.AbstractMethodError: javax.xml.parsers.DocumentBuilderFactory.setFeature(Ljava/lang/String;Z)V

at gov.nasa.worldwind.util.WWXML.createDocumentBuilder(Unknown Source)

at gov.nasa.worldwind.util.WWXML.openDocumentStream(Unknown Source)

at gov.nasa.worldwind.util.WWXML.openDocumentStream(Unknown Source)

at gov.nasa.worldwind.util.WWXML.openDocumentFile(Unknown Source)

at gov.nasa.worldwind.util.WWXML.openDocument(Unknown Source)

at gov.nasa.worldwind.Configuration.loadConfigDoc(Unknown Source)

at gov.nasa.worldwind.Configuration.<init>(Unknown Source)

at gov.nasa.worldwind.Configuration.<clinit>(Unknown Source)

at gov.nasa.worldwind.util.Logging.<clinit>(Unknown Source)

at gov.nasa.worldwind.formats.vpf.VPFLibrary.fromFile(Unknown Source)

at gov.nasa.worldwind.formats.vpf.VPFUtils.readLibrary(Unknown Source)

at gov.nasa.worldwind.formats.vpf.VPFDatabase.createLibraries(Unknown Source)

at gov.nasa.worldwind.formats.vpf.VPFDatabase.fromFile(Unknown Source)

at com.metsci.glimpse.dnc.convert.Vpf2Flat.readVpfDatabase(Vpf2Flat.java:251)

at TaisrSPWorkbench.Vpf2FlatConverter.lambda$null$17(Vpf2FlatConverter.java:354)

at java.lang.Thread.run(Thread.java:745)

Any ideas?

Ryan

[Ryan Jones]

I am also getting this error when using WorldWind by itself. I am going to try 2.4 again.

[Mike Hogye]

Both Glimpse versions use the same WW version; if the issue shows up with WW by itself, then the Glimpse version shouldn't matter. Doesn't hurt to try anyway, though.

You have a set of VPF dirs that worked for you in the past, right? Does Vpf2FlatConverter work okay on that VPF data?

My guess is that it's something about the VPF data -- dir structure, or something. Downloading from NGA, and unzipping into the right dir structure, is the most error-prone part of the whole process. It's probably worth double-checking that -- maybe compare the dir structure of DNC13 to the dir structure of the data that worked in the past.

[Mike Hogye]

Maybe future versions of Vpf2FlatConverter should be able to do the download-and-extract steps automatically.

[Ryan Jones]

I think this issue has something more to do with my netbeans configuration. I moved the project over to another computer and now have this issue. Running it on the original computer doesn't display the issue. 

-Ryan

[Mike Hogye]

If you figure it out, we'd be interested to hear what was going on -- maybe we can do something to avoid the issue in the future.

[Ryan Jones]

The issue ended up being that on one computer I had created a library that had all the jar files from JIDE. This included xerces.jar and this is what was causing my issue. I removed it and it works fine. No more exceptions while running either! I do get some messages that I am unsure of. I believe it is related to Glimpse but can't be 100% positive. Just wondering if you have run into it.

line 14:28 no viable alternative at character '['

line 14:29 no viable alternative at character ']'

line 17:26 no viable alternative at character '['

line 17:27 no viable alternative at character ']'

line 20:26 no viable alternative at character '['

line 20:27 no viable alternative at character ']'

line 14:28 no viable alternative at character '['

line 14:29 no viable alternative at character ']'

line 17:26 no viable alternative at character '['

line 17:27 no viable alternative at character ']'

line 20:26 no viable alternative at character '['

line 20:27 no viable alternative at character ']'

No other info and it doesn't appear to effect anything so I have been ignoring it but everytime someone uses my program and encounter a bug they instantly mention these lines so I would like to get rid of them. ;)

-Ryan

[Mike Hogye]

Cool, thanks for passing along the answer.

That error message isn't familiar ... but it looks like it probably comes from ANTLR, which Glimpse depends on. Maybe another version of antlr-runtime.jar is ending up on the classpath? (Glimpse dependency is antlr-runtime 3.4.)

[Geoffrey Ulman]

I have seen that error before. So I don't think it's related to another version of the antlr-runtime on the classpath.

I'm guessing one of the shaders Glimpse uses is generating those errors, so you only see them if you're using that shader.

I'll try to narrow down which one it is.

(Although also we'd like to move away from the custom ANTLR parsing of GLSL shader code that we're currently doing anyway).

[Geoffrey Ulman]

Looks like the issue was indeed ANTRL and was in a shader used by IconPainter.

Fix was straightforward. It'll be included in the next release. I'll try to do a 2.5.2 minor bug fix release soon.

[Ryan Jones]

That makes since since I just recently started using that. Thanks for all the quick responses and fixes.

-Ryan