[osg-users] Convert .earth file to .osg format

[Steven Jones]

Hope someone can help with this:

I need to convert an earth file to the .osg format. I've not found any solid info yet as to how this can be done, or if it can be done at all. The 3rd party program this is for can load .osg or .ive GIS maps.

I've tried osgconv, and it creates a new file with the .osg extension, but it is a zero-byte file (no error messages are generated).

Many thanks in advance!

Cheers,
Steven

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=75875#75875





_______________________________________________
osg-users mailing list
osg-...@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

[Chris Hanson]

So, the .earth format is not really a geometry format. It's a descriptor of a special node data file type used by the osgEarth library, and not representable in a .osg file.

You can't use a .earth file without the osgEarth library. If you HAVE the osgEarth library, you can already load .earth and don't need to convert it to .osg.

So, tell us what you're actually trying to DO, not the method by which you're trying to accomplish it, and maybe we can help.

--

Chris 'Xenon' Hanson, omo sanza lettere. Xe...@AlphaPixel.com http://www.alphapixel.com/

Training • Consulting • Contracting

3D • Scene Graphs (Open Scene Graph/OSG) • OpenGL 2 • OpenGL 3 • OpenGL 4 • GLSL • OpenGL ES 1 • OpenGL ES 2 • OpenCL

Legal/IP • Forensics • Imaging • UAVs • GIS • GPS • osgEarth • Terrain • Telemetry • Cryptography • LIDAR • Embedded • Mobile • iPhone/iPad/iOS • Android

@alphapixel facebook.com/alphapixel (775) 623-PIXL [7495]

[Steven Jones]

Chris - Thanks for the answer. I do understand the nuts & bolts of the OSG architecture, but was looking for a way to further streamline loading of a very large quantity of GIS imagery and elevation data that are referenced within the .earth file (over 4,000 images and close to 1,000 elev files).

They are all GeoTiffs, and I've already run the compression tools (GDAL) to reduce the size of the imagery to increase performance/reduce load time.

Given its size, it's likely that the .earth file is going to be the best/simplest option overall.

On a side note, I'm seeing the borders around all of the tiles when I view this terrain database in osgearth_viewer, and I'm not sure how to eliminate them. They are all very fine black lines, and I'm sure the georeferencing for each tile/GEOTiff is correct. Hopefully there's a way to remove them through a setting in the .earth file, but I've been unsuccessful so far.

...

Cheers,
Steven

------------------
Read this topic online here:

http://forum.openscenegraph.org/viewtopic.php?p=75879#75879

[Chris Hanson]

You may need to play with layering and other things to get the results you want at the borders.

Alternately, have you considered if your work could be built through Virtual Planet Builder, which DOES actually just produce "dumb" .osg (actually, .osgb is recommended) files?

The result might be larger, but simpler/faster depending on how complex your layering is.

[Steven Jones]

I'd considered VPB previously, but now it looks like it may be the most practical solution to meet our needs. Wish me luck, and thanks again Chris!

Cheers,

Steven

------------------
Read this topic online here:

http://forum.openscenegraph.org/viewtopic.php?p=75897#75897

[Chris Hanson]

You're welcome. It may also be conceivable to use osgearth tools to produce a composite tilemap set to feed to VPB, giving you some of the best of both worlds. 

[Steven Jones]

Chris - Is there a tutorial of some sort that would cover the basic steps to begin building a database in VPB using GeoTiff elevation and imagery data for the input?

I've downloaded and compiled VPB (MS Windows), but since it's mainly command-line driven, it's not all that intuitive. Also, most of what I've found online deals with modifying an existing database (that already has an .osgb master file).

Any help/tips/tricks would be greatly appreciated!

Cheers,

Steven

------------------
Read this topic online here:

http://forum.openscenegraph.org/viewtopic.php?p=75900#75900

[Chris Hanson]

I don't have a ready to consume tutorial, but your first goal is to use osgEarth to build a dataset by merging all your data together.

The second is to feed that to VPB.

Let me think on the matter a bit.

[Steven Jones]

Hi Scott,

Are you referring to a TMS dataset? Or something different? (All of my elevation and imagery files are tiled GeoTiffs.)

Thanks!

Steven

------------------
Read this topic online here:

http://forum.openscenegraph.org/viewtopic.php?p=75903#75903

[Chris Hanson]

Basically, osgEarth is really good at doing fancy layering of data sources. VPB is a little trickier, but can do most of the same things (for elevation and raster data anyway).

I was thinking it might be easier to use osgEarth to turn your multiple layers of GeoTIFF images into a single elevation layer and a single imagery layer, and just feed that to VPB.

But, I think really, feeding the original data directly to VPB is better because it lets VPB use its own intelligence about the different layers' resolution.

So, do you know how to build a model using VPB with multiple input layers? Here's some info with a few sample command lines:

http://alphapixel.com/wp-content/uploads/2015/04/OpenSceneGraph-OSG-Virtual-Planet-Builder-VPB-OSGDEM.pdf 

 

[Steven Jones]

I'd downloaded that guide before and briefly reviewed it, but that was when I first installed VPB but hadn't yet tried it.

It would certainly be much more efficient to combine the imagery and elevation data and create an osg or ive formatted database.

But, since I have so many GeoTiffs and elevation files, is it possible to just point to the specific folder for each type on the command line?

For example, here's the general structure for the imagery:

C:\DB\N33\W118\004_Imagery

The imagery files in this directory are L0 through L08

Each of these sub-directories contain multiple sub-sub-directories, named U0 through U"X" (some have 20 or more folders).

File names follow this naming convention:

N33W118_D004_S001_T001_L03_U0_R5a.tif
N33W118_D004_S001_T001_L03_U0_R6a.tif etc...

The structure is basically the same for the elevation data:

C:\DB\N33\W118\001_Elevation

The overall database format follows what is common for the CDB format.

The document you referenced shows listing of individual files to be merged/combined - Pointing to the overall sub-directories would be much easier, if that's possible...


...

Thank you!

Cheers,
Steven

------------------
Read this topic online here:

http://forum.openscenegraph.org/viewtopic.php?p=75908#75908

[Robert Osfield]

Hi Steven,

It's probably getting on decade since I last used or did any serious
worked on VPB so can't recall the specific command line options to use
off the top of my head. So will just provide some general notes that
may help.

It's possible to have osgdem (single process build) and vpbmaster
(multi-process distributed build) using a directory as input rather
than individual files. Run osgdem -h to get the command line options.
You can generate a single source file from your initial input
parameters and reuse that source file subsequently - this is what is
also used in the distributed build.

I'm afraid I can't help beyond this without sitting down at my
workstation to do some test builds after doing re-learning of the
command line options.

FYI, in the coming months I'll be doing some work on paging in for the
VulkanSceneGraph project and will likely do some work with VPB to get
it to generate native .vsgb rather than .osgb files. At this point
I'll need to dive back into VPB and re-lean how to use it properly :-)

Robert.

[Chris Hanson]

Also, to add to that, osgdem relies heavily on GDAL to ingest raster data. GDAL has several tricks for combining multiple files or directories and referencing them as one dataset. For example:

https://gdal.org/gdal_vrttut.html  

I don't recall ever using that technique with VPB, but in theory it could be helpful.

[Steven Jones]

Chris,

I've been trying osgdem to build the database, and pointing to the overall imagery and elevation sub-directories works fine.

I receive no error messages, and the build completes, but when I use osgviewer to inspect the .ive file, it is completely flat - no visible change in elevation - terrain resolution is perfect though. (When I view the .earth file in osgearth_viewer, the terrain's elevation is rendered correctly.)

Below is the command line that I've used to build the database - I've changed the LOD level to various settings, even up to 8, and tried other settings based on osgdem's -h info (such as --PagedLOD), but no joy.

osgdem --geocentric -t C:\DB\N33\W118\004_Imagery -d C:\DB\N33\W118\001_Elevation -l 8 -o db.ive

Also, if I create the database as .osg instead of .ive, osgviewer throws the following error: "Problem, no locator found in any of the terrain layers."

Maybe I'm missing something obvious, but for the life of me I can't get past this point... ;-( (It is very important that the database render the elevation in a 3D format.)

One other item of interest is that neither --interpolate-imagery nor --interpolate-terrain work in osgDem... They both throw errors that they are unrecognized commands. The version of VPB/osgDem I'm running is 0.9.11 (I was trying different options to generate the database when I encountered this.)

...

Cheers,

Steven

------------------
Read this topic online here:

http://forum.openscenegraph.org/viewtopic.php?p=75915#75915

[Chris Hanson]

On Mon, Apr 29, 2019 at 9:09 PM Steven Jones <steve...@gmail.com> wrote:

I receive no error messages, and the build completes, but when I use osgviewer to inspect the .ive file, it is completely flat - no visible change in elevation - terrain resolution is perfect though.

  Is the imagery in place and proper resolution?

 

Below is the command line that I've used to build the database - I've changed the LOD level to various settings, even up to 8, and tried other settings based on osgdem's -h info (such as --PagedLOD), but no joy.
osgdem --geocentric -t C:\DB\N33\W118\004_Imagery -d C:\DB\N33\W118\001_Elevation -l 8 -o db.ive

  What elevation units is your elevation data in?

  Can you verify that OSG or osgDEM is able to read it ok otherwise? Do you have any other known-good sample data to throw at it to see if your elevation file values just aren't being understood correctly?

 

Also, if I create the database as .osg instead of .ive, osgviewer throws the following error:  "Problem, no locator found in any of the terrain layers."

  Is your OSG even built with the deprecated .osg format plugin?

  You can save as the more modern .osgb (binary) and .osgt (text) formats instead of .osg and .ive. They are preferred now, though I haven't tried them with VPB/osgdem myself.

  If you can get it into .osg or .osgt format -- or use osgconv to convert the successful .ive file(s) to .osg/.osgt you might be able to examine the data values in the created rasters and see if there are all zeros or just some VERY small values.

 

Maybe I'm missing something obvious, but for the life of me I can't get past this point... ;-(   (It is very important that the database render the elevation in a 3D format.)
One other item of interest is that neither --interpolate-imagery nor --interpolate-terrain work in osgDem... They both throw errors that they are unrecognized commands. The version of VPB/osgDem I'm running is 0.9.11 (I was trying different options to generate the database when I encountered this.)

  Huh. I can't recall if I ever used them. You'd have to look into the source to see if they're even there or if they're able to be made to work.

 

[Glenn Waldron]

Steven,

You should consider making a GDAL VRT. It's a format for compositing source files together into one virtual data source. You can build it by hand or use the gdalbuildvrt tool.

https://www.gdal.org/gdalbuildvrt.html  

Glenn Waldron / osgEarth

[Steven Jones]

Chris,

When I examine an image GeoTiff with GDALinfo, here's what is output:

Driver: GTiff/GeoTIFF
Files: N33W118_D004_S001_T001_L00_U0_R0a.tif
Size is 1024, 1024
Coordinate System is:
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433],
AUTHORITY["EPSG","4326"]]
Origin = (-118.000000000000000,34.000000000000000)
Pixel Size = (0.000976562500000,-0.000976562500000)
Metadata:
AREA_OR_POINT=Area
TIFFTAG_SOFTWARE=GeoJasPer <www.dimin.net>
Image Structure Metadata:
COMPRESSION=JPEG
INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left (-118.0000000, 34.0000000) (118d 0' 0.00"W, 34d 0' 0.00"N)
Lower Left (-118.0000000, 33.0000000) (118d 0' 0.00"W, 33d 0' 0.00"N)
Upper Right (-117.0000000, 34.0000000) (117d 0' 0.00"W, 34d 0' 0.00"N)
Lower Right (-117.0000000, 33.0000000) (117d 0' 0.00"W, 33d 0' 0.00"N)
Center (-117.5000000, 33.5000000) (117d30' 0.00"W, 33d30' 0.00"N)
Band 1 Block=256x256 Type=Byte, ColorInterp=Red
Band 2 Block=256x256 Type=Byte, ColorInterp=Green
Band 3 Block=256x256 Type=Byte, ColorInterp=Blue

This is one of the GeoTiff elevation files:

Driver: GTiff/GeoTIFF
Files: N33W118_D001_S001_T001_L00_U0_R0.tif
Size is 1024, 1024
Coordinate System is:
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433],
AUTHORITY["EPSG","4326"]]
Origin = (-118.000488281250000,33.999511718750000)
Pixel Size = (0.000976562500000,-0.000976562500000)
Metadata:
AREA_OR_POINT=Point
Image Structure Metadata:
INTERLEAVE=BAND
Corner Coordinates:
Upper Left (-118.0004883, 33.9995117) (118d 0' 1.76"W, 33d59'58.24"N)
Lower Left (-118.0004883, 32.9995117) (118d 0' 1.76"W, 32d59'58.24"N)
Upper Right (-117.0004883, 33.9995117) (117d 0' 1.76"W, 33d59'58.24"N)
Lower Right (-117.0004883, 32.9995117) (117d 0' 1.76"W, 32d59'58.24"N)
Center (-117.5004883, 33.4995117) (117d30' 1.76"W, 33d29'58.24"N)
Band 1 Block=1024x1 Type=Float32, ColorInterp=Gray

I don't see the measurement value type, but I'm fairly sure they are both in meters. (I also confirmed the above data in QGIS.)

I'll also look into gdalbuildvrt - Thanks!

Steven

------------------
Read this topic online here:

http://forum.openscenegraph.org/viewtopic.php?p=75920#75920

[Chris Hanson]

Band 1 Block=1024x1 Type=Float32, ColorInterp=Gray

 don't see the measurement value type, but I'm fairly sure they are both in meters. (I also confirmed the above data in QGIS.)

This seems plausible. Did you check to see what value range this elevation TIFF has?

I can't see anything obviously wrong.

 

[Steven Jones]

Chris,

Not sure what value range you are referring to (still learning all I can regarding the GeoTiff file structures).

"This seems plausible. Did you check to see what value range this elevation TIFF has?"

Steven

------------------
Read this topic online here:

http://forum.openscenegraph.org/viewtopic.php?p=75930#75930

[Chris Hanson]

It's a single-precision 32-bit floating point TIFF. Values could be anywhere from -FLT_MAX to FLT_MAX.

I'm just making sure it doesn't have weird values like sea level being 0 and Everest being 8.8 (in units of kilometers).

[Steven Jones]

Chris,

Chris Hanson wrote:
> It's a single-precision 32-bit floating point TIFF. Values could be anywhere from -FLT_MAX to FLT_MAX.
>

> I've tried several methods to try to find the FLT_MAX values but so far no joy, so I have no idea what it could be set to... Can you recommend a method to dig this info out?

>
> Steven
>
> ------------------
> Read this topic online here:

> http://forum.openscenegraph.org/viewtopic.php?p=75930#75930 (http://forum.openscenegraph.org/viewtopic.php?p=75930#75930)
>
>
>
>
>
> _______________________________________________
> osg-users mailing list
> ()
> http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org (http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org)
>

--
Chris 'Xenon' Hanson, omo sanza lettere. http://www.alphapixel.com/ (http://www.alphapixel.com/)

Training • Consulting • Contracting
3D • Scene Graphs (Open Scene Graph/OSG) • OpenGL 2 • OpenGL 3 • OpenGL 4 • GLSL • OpenGL ES 1 • OpenGL ES 2 • OpenCL
Legal/IP • Forensics • Imaging • UAVs • GIS • GPS • osgEarth • Terrain • Telemetry • Cryptography • LIDAR • Embedded • Mobile • iPhone/iPad/iOS • Android

@alphapixel (https://twitter.com/alphapixel) facebook.com/alphapixel (http://facebook.com/alphapixel) (775) 623-PIXL [7495]

------------------
Post generated by Mail2Forum[/quote]

------------------
Read this topic online here:

http://forum.openscenegraph.org/viewtopic.php?p=75935#75935

[Chris Hanson]

I would think QGIS would do it. There may be other, simpler tools too.

Chris 'Xenon' Hanson, omo sanza lettere. Xe...@AlphaPixel.com http://www.alphapixel.com/

Training • Consulting • Contracting

3D • Scene Graphs (Open Scene Graph/OSG) • OpenGL 2 • OpenGL 3 • OpenGL 4 • GLSL • OpenGL ES 1 • OpenGL ES 2 • OpenCL

Legal/IP • Forensics • Imaging • UAVs • GIS • GPS • osgEarth • Terrain • Telemetry • Cryptography • LIDAR • Embedded • Mobile • iPhone/iPad/iOS • Android

@alphapixel facebook.com/alphapixel (775) 623-PIXL [7495]

[Steven Jones]

Sorry to be a bother Chris, but I've drilled through QGIS' various methods of viewing GeoTiff metadata but cannot find anything referencing any values named "Flt_Max". I must be missing something...

...

Cheers,

Steven

------------------
Read this topic online here:

http://forum.openscenegraph.org/viewtopic.php?p=75947#75947

[Chris Djali]

Hi,

I think they're talking about the C preprocessor definition, FLT_MAX rather than anything specific to anything used here:

https://www.tutorialspoint.com/c_standard_library/float_h.htm

It's the maximum value of an IEEE 32-bit float.

Cheers,
Chris

------------------
Read this topic online here:

http://forum.openscenegraph.org/viewtopic.php?p=75948#75948

[Steven Jones]

Hi Chris "D",

Thanks - I'd done some researching and came to the same basic conclusion... For now, I'm going to trust that the values are as they should be...

I've created a VRT file using gdalbuildvrt but am not sure how that can be used now to ultimately create an osg (or osgb) file to use within my application to view it in 3D.

The files used to build the vrt were both elevation and imagery GeoTiffs, and I used the -separate option (tried it without and after a little over 1,000 files were processed, gdalbuildvrt stopped due to too many errors. (The mosaic has 600+ elevation files and 4,000+ imagery files - error was likely due to differences related to the number of bands and float versus int.)

How best to proceed?

...

Cheers,

Steven

------------------
Read this topic online here:

http://forum.openscenegraph.org/viewtopic.php?p=75949#75949

[Chris Hanson]

I really feel like you need to step back a couple of steps and get a working process.

Like, can you build a dataset with JUST elevation data and just ONE simple GeoTIFF terrain elevation file? When you have a process that works, you can work forward, adding complexity and watch for where it breaks and you'll know what the problematic aspect is.

At the moment, the problem could be anywhere.

[Steven Jones]

Chris,

I can definitely do that, since I have the original files (imagery and elevation) in segregated in separate sub-directories.

I'd still need to be able to visualize them though as I test the build process to ensure that I'm seeing the 3D aspect of the overall terrain. With a VRT, I'm not sure how to do that.

...

Cheers,

Steven

------------------
Read this topic online here:

http://forum.openscenegraph.org/viewtopic.php?p=75951#75951

[Chris Hanson]

WHen I say build a dataset, I mean, run it through osgdem and produce a .osg or .osgb and test it.

Start simple, then if you get that working, add complexity until you find the breaking point, then fix whatever you identify as being the problem.