WARNING - INVALID COVARIANCE MATRIX

[Dennis Owen]

Hello, I continue to get the aforementioned error and cannot seem to isolate the cause.

I am working with data that I have attempted to bring in through the Trimble Data Exchange converter, but was unable due to parser errors.

I was able to convert the data using Microsurvey Starnet then formatted that data to the LSA format (attached).

If I proceed with the LSA analysis, i get the following error.

------------ iteration 1 ----------------

# Solve the least squares problem at iteration 1

 Error - Singular measurement covariance: text 0:Non-positive eigenvalue -1.889629e-03 at col 2: lowerCholesky() requires positive-definite input

lsasolver is terminating with code 15: There is a singular measurement covariance.

Any help that anyone may be able to provide would be much appreciated.

Thank you

[Clark Hughes]

Howdy Dennis,

So the issue seems to be that the covariance matrices for some/all of those measurements aren't physical; they're not positive definite.  Observe the negative diagonal terms, e.g., in:

DXYZ 1 27 602.53536 -674.60419 -915.62454 m ...

         1.260e-05          1.220e-05          8.360e-05 ...

                           -1.180e-05          9.400e-05 ...

                                              -7.200e-05

I know we have users with Trimble GNSS hardware who have workflows compatible with SALSA.  If you want to attach an example file maybe someone on this channel can take a look and try to help you address the issue upstream, since apparently the Starnet conversion is not helpful.

Regards,

Clark

[Dennis Owen]

Thank you for the quick reply.  I have attached the RAW Trimble Data Exchange File that I am having trouble running through the converter.  Note that our survey was conducted in US Survey feet, and I am assuming that this is the reason for the parse issue.  When I try to import the RAW file I get the following errors:

__________________________________________________________________________________________

PARSE WARNING: 2021-049_TDExchange_RAW_USft.lsa(0): Could not open included file: 2021-049_TDExchange_RAW_USft.lsa

___________________________________________________________________________________________

The resulting lsa file is never created from the converter.

To rectify the parse warning, I have attempted to convert the data to meters, remove entries that seem to be ignored by the converter after reviewing the code, change the leading "Vector=1" to an indexing tag, and delete the second line item "?" since the code is looking for the fromPoint term in line item [2].  When I try to import the modified version, I get the following errors:

____________________________________________________________________________________________

ERRORS ENCOUNTERED:

Traceback (most recent call last):

File "C:/Program Files/SALSA/scripts/converters/converterlauncher.py", line 114, in <module>

lsaConverter.convert()

File "C:\Program Files\SALSA\scripts\converters\TrimbleToLSA.py", line 611, in convert

self.writeOutputFile()

File "C:\Program Files\SALSA\scripts\converters\TrimbleToLSA.py", line 565, in writeOutputFile

self.writeOutLSA()

File "C:\Program Files\SALSA\scripts\converters\TrimbleToLSA.py", line 432, in writeOutLSA

gps_exists = self.parseDXYZ()

File "C:\Program Files\SALSA\scripts\converters\TrimbleToLSA.py", line 169, in parseDXYZ

currentLine = next(lineItr)

StopIteration

________________________________________________________________________________________________

This is where I attempted to convert the file using Starnet.  I have been using Starnet for lsa adjustments to date and after coming across your software I wanted to see if I could switch.

Thanks again for taking a look.

[Corwin Salsa]

Hi,

Thanks for providing the Trimble file you're having issues with. We'll try to take a look at some point to see if we can identify any bugs in the converter. Thanks for reaching out, always good to hear about problems you experience when running SALSA and it's associated import/export scripts. We'll let ya know what we find.

Corwin

[Rob Milburn]

Dennis,

Thanks for reaching out and providing us with all of the information necessary to investigate this issue. During the course of our investigation, we discovered that there were two different components to your observations, an issue with the converter parsing different units and a validation step SALSA performs on covariance matrices. I'll speak to the parsing issue first.

As we were working with your 2021-049_TDExchange_RAW_USft.asc file, we encountered parsing errors from our Trimble Data Exchange converter as well. After investigating our TDEF converter script, we came to realize that we did not provide proper support to handle units of "US survey feet"  and "International feet" from Trimble. Given that SALSA supports units of feet for linear measurements as well as DXYZ records, we decided to extend our TDEF converter's functionality to write out in units of feet as well. We have recently incorporated this fix and will have it out as part of our 1.14.0 release in the near future.

After we discovered the issue with the converter not parsing different units, we took another look at the covariance warnings SALSA was providing. For the 2021-049_TDExchange_RAW_m.asc file, we believe the covariance warnings we encountered are most likely due to the removal of one of the columns. The TDEF converter assumes a more strict format based on TDEF specifications, thus removing a field will result in covariance elements not being mapped appropriately. However, after implementing the fix to our TDEF converter, the 2021-049_TDExchange_RAW_USft.asc file still receives covariance matrix warnings after being imported into SALSA. After some debugging, it appears that the matrices are not positive semi-definite. The matrices seem properly parsed based on the converter's mapping. The initial lsa file you provided seems to have a similar problem as well. For the sake of clarity, was the 2021-049_TDExchange_RAW_USft.asc file grabbed straight from the instrument or did it go through some preprocessing prior to being passed as input into SALSA? If the answer is the latter, we can investigate the instrument file to see if it still produces the same warning. If it's the former, unfortunately there is not much we can do as those covariance matrices are nonphysical.

Thanks again for all of the detail provided. The correspondence helped highlight a bug that we missed during our implementation of the TDEF converter script. We hope the patch in our next release will help facilitate your workflows in any future projects that utilize SALSA.

[Dennis Owen]

Rob,

Thank you for taking the time to investigate this issue.

To give you the full context, I had two survey crew working simultaneously on this job to start and therefore I have two files from their collectors to work from.  The first round of observations of the control was done by field crew A using Spectra Precision data collector and SP80 GPS units.  I have attached the native file as 2021-049 Round 1.JXL The second crew conducted Rounds 2 and 3 of the observations using a Trimble collector and Trimble R8 GPS units.  I have attached this native file as 2021-049 Rounds 2_3.job.

I then used the Trimble ASCII File Generator to export a Trimble Data Exchange Format (.asc) from each of the files.  Next, I combined the observations into the 2021-049_TDExchange_RAW_USft.asc. and remapped (renumbered) the observations to the correct "from : to" points (mostly changed the "to" points as I plan to hold the coordinates of Point 1 as fixed).  Finally, I removed the extraneous [Stations], [Keyed In Coordinates ], and observations that weren't related to the control points.

Please let me know if you have any further questions or suggestions as I am happy to help out.

[Dennis Owen]

Sorry, I am having trouble posting the message with the attachments.

[Dennis Owen]

Second attachment attempt. The .job file was too large to attach without zipping.