Cesium 1.55+ requests tiles at zoom levels greater than our max zoom level, causing 404 errors

[abhi...@gmail.com]

We use a custom terrain provider (Heightmap terrain format) that goes up to zoom level 10. In Cesium 1.54 and below, this seems to work fine. When we zoom in very close to the earth, Cesium only requests terrain tiles from the max of zoom level 10.

However, in cesium 1.55 and beyond, it looks like Cesium is requesting zoom levels that do not exist in our tile set (i.e zoom level 20+). This results in many 404 errors being thrown in the console, followed by messages like this:

"An error occurred in "CesiumTerrainProvider": Failed to obtain terrain tile X: 2429379 Y: 1120026 Level: 22."

I know that Cesium 1.55 included an overhaul on how terrain tiles are loaded, but I'm not sure why it seems to be requesting deeper zoom levels in this version.

Should we be specifying something differently in our TerrainProvider to prevent it from requesting past zoom level 10 in version 1.55? Is the issue related to us using the Heightmap terrain format instead of Quantized mesh?

[Omar Shehata]

This sounds like a bug to me. Are you able to provide a Sandcastle example with a sample of your data that shows this issue? Then it would be good to open a GitHub issue for it.

This is the pull request that made the overhaul:

https://github.com/AnalyticalGraphicsInc/cesium/pull/7061

It looks like not much has changed in the CesiumTerrainProvider.prototype.getTileDataAvailable though.

I wonder if it's because the new system will skip levels in order to speed up terrain loading, so it might be skipping too far? Can you try setting globe.loadingDescendentLimit to 0 to see if that fixes it?

https://cesiumjs.org/Cesium/Build/Documentation/Globe.html?classFilter=Globe#loadingDescendantLimit

[abhi...@gmail.com]

Hi Omar,
Thanks for the response. Unfortunately, setting our globe's "loadingDescendantLimit" to 0 did not resolve the issue.

I can't send a sandcastle example demonstrating the issue, since my main example involves some internal terrain data. That said, our data is of Heightmap format. Do you think it has anything to do with that?

[Omar Shehata]

Just to confirm, was your terrain generated by uploading it to Cesium ion? If not, are you able to describe or share a sample snippet of how you're loading it?

If you're adding it as a CesiumTerrainProvider, I would try to debug this function:

https://github.com/AnalyticalGraphicsInc/cesium/blob/master/Source/Core/CesiumTerrainProvider.js#L655

Specifically where it gets the layer which has information on how many levels are available in this tileset.

[Abhi Sundu]

We're using CesiumTerrainProvider with a URL.

For example:

var viewer = new Cesium.Viewer('cesiumContainer');

viewer.terrainProvider = new Cesium.CesiumTerrainProvider({

  url: 'https://myterrain.com/terraindata/'

});

Thanks for the tip about stepping through that function. I walked up the call stack from that function and one difference that I see is

- In Cesium 1.54 (where we do not experience this issue)

In GlobeSurfaceTile.js, in the function "processTerrainStateMachine", it looks like it's checking if the tile's "loadedTerrain" property exists. If it doesn't, then it seems to do some upsampling.

This also seems to allow it to not request tiles past our max zoom level (i.e when a tile with level = 19 enters this function, it doesn't try to load it thanks to checking "if (defined(loaded))" (line 212792 of Cesium.js in 1.54).

Only tiles that pass this check seem to go to "processLoadStateMachine" which then calls "requestTileGeometry".

- in Cesium 1.55 and 1.56.1 (where we experience the issue)

In GlobeSurfaceTile.js, in the same function "processTerrainStateMachine", it looks like it just goes straight to "requestTileGeometry" if the terrainState is "UNLOADED". If it does this with a tile that has too high a zoom level, it throws the error from before. There's a check to see the tile is FAILED, and if so it upsamples the tile (which is what I think 1.54 does when the tile is not "loaded"), but for some reason our terrain doesn't return FAILED when being requested for zoom levels past our max zoom level.

Is there something we can do to tell these functions to not request tiles past our max zoom level of 10? We're adverse to making changes in the Cesium source; if there's an API we can use to do this outside the cesium source that would be really good.

Thanks,

Abhi

[Omar Shehata]

The getTileDataAvailable function is what's responsible for checking that a tile/level is not available and not making the request:

https://github.com/AnalyticalGraphicsInc/cesium/blob/master/Source/Core/CesiumTerrainProvider.js#L944

One quick thing could be to add a check there for what you know is the maximum level. If you're trying to avoid editing the source I wonder if you can inspect the _availability and override the maximum level variable it's using there.

The other thing is that the level 10 tiles should have the childMask on the data they return set to 0, which communicates that it's the leaf tile. 

[Abhi Sundu]

Thanks for the response and for the tip about the "_availability" property. In our app, it looks like this property is undefined by default. I tried editing the "_availability" property to set the maximumLevel to 10 and it seems to have worked, but I also had to override the "isTileAvailable" function and the "_hasMetadata" property to avoid errors. I don't feel so good about overriding the "isTileAvailable" function, is there a way I can just override the maximumLevel property?

[Abhi Sundu]

Also: I have a repro now, we found an app (not ours) that uses terrain that can reproduce the issue.

In the following sandcastle, navigate to the cesium logo billboard and go to the console. You'll see a ton of "failed to obtain terrain" errors.

This next sandcastle is the same code but in cesium 1.54 instead where we do not observe the issue (make sure to change the baseman to something not bing-related). No terrain console errors should appear in this case. (I do not own the terrain in these sandcastle examples.)

I created a github issue for this, here's a link.

One other issue we're running into with the "_availability" workaround you described is: in some cases, the terrain we use has a variable max zoom level. Part of our terrain will have a max zoom level of say 12, and the rest may have a significantly lower zoom level. If we specify "12" as the max level of "_availability", then we receive errors anywhere on the globe where we don't have a max zoom level of 12. This part is also new with version 1.55. 

Let me know if you have any ideas, thanks.

Abhi

[Omar Shehata]

Thanks again for your reporting this and for all your help debugging it Abhi! For anyone watching this thread, this issue is now fixed with this PR:

https://github.com/AnalyticalGraphicsInc/cesium/pull/7851