On Oct 19, 2014, at 4:49 AM, Martin Isenburg <
martin....@gmail.com> wrote:
> Hello Thomas,
>
> maybe having more than one LAZ implementation will actually "strengthen" the LAZ format to ultimately become an officially adopted standard (ASPRS? ISPRS? OGC?) because it is then no longer solely seen as "Martin's format". I agree your worry that having the emscripten/HoBu laz-perf version drift out of sync with the rapidlasso LASzip version would be a horrible thing to happen. But the wide-spread support that LAZ has today has a lot to do with Howard's efforts to promote it via libLAS and secure funding for the first open source release via the Army Corps of Engineers (see "Gold Sponsors" on
http://laszip.org). Given that the two teams who are working on independent LAZ libraries are exactly those who have invested many years of efforts into seeing LAZ succeed as a standard should ease your worries somewhat.
This may be true, but it doesn't prevent another party from coming in without the same intention and desire to get along and go with the flow. You are correct that I am very interested in not disrupting LAZ with our efforts to get it working in Javascript, but at the same time, I'm disappointed by your disinterest in moving forward with our efforts as LASzip software. Our effort is better engineering (works on ARM, JavaScript, benchmarks faster, more flexible, modern C++) and it is byte-compatible with current LASzip. We put a lot into it. It is frustrating to have to chose between the burden of maintaining a separate fork or trying to go back and put our square peg (Javascript support) in a round hole (current LASzip codebase) (tried and failed).
Another frustrating aspect of LASzip as a second banana implementer is you've been able to permute the wire format at your leisure. This is just fine if it's only you who has to worry about it, but now that there is at least one other implementation, what are we supposed to do with regard to things like your 1.4 compatibility or potentially baking in spatial sorting to the format proper? Again, we've chosen to follow along, but this posture is a hindrance. Do we get to have a say on how such things are to be implemented? Design principles that should be followed? Such are the travails of software formats and their implementers...
Government agencies like the ones Thomas represents get very uneasy with the idea of community-based formats because it is easy for this community to be disrupted in intentional and unintentional ways. True standards, in a true standards body with real release, development, and conflict procedures make those agencies feel more comfortable about mandating formats in contracting language. Because these organizations and/or their derivatives are funding the collection of a majority of the LiDAR data at this time, they have an outsized impact in determining formats of the entire ecosystem.
In that regard, I think LAZ has wildly outperformed an expected market adoption due to its fantastic achievement of its stated task. It works awesome and it is free -- this makes it very difficult for a commercial vendor to come along and try to sell a better mouse trap. A government agency, however, can feel comfort in mandating a closed format due to the fact that they have only one body to deal with to achieve its goals. An open standard (not simply a specification) has the same properties -- a single entity -- that provides that same comfort. A community standard, with slightly incompatible implementations and no body to evolve the format beyond the needs of commercial interests who are its gatekeepers, causes anxiety even if there is no real reason to be nervous as you've described above.
This is the sociological aspect of zLAS that LAZ hasn't addressed. It isn't simply enough to tackle every real and imagined technological challenge that ESRI's format introduces. It's also the fact that USGS or OS (UK) or CountryGovernmentDataAgency don't have to herd us cats into getting the format to respond to their data archival needs which they then turn into contracting language. Now that we have at least two different-but-derived implementations, we get to test whether or not the LAZ ecosystem better preserves its past in compatible ways. LASzip has done a perfect job thus far, but that's only one piece of software with one set of commercial priorities tugging on it. Now there's at least one more...
I'll note that I have lots of experience with another community-based format -- I am an author of the GeoJSON specification. We have achieved great success with the format by being supremely stubborn in never changing it. It has not evolved in six years since the first document was released. It has plenty of warts, is terribly inefficient, redundant, and plodding. We've lost count of the ways we've been told we're idiots. Indeed, many have made much better mouse traps, but GeoJSON is the one that has had the widest penetration due to the fact that it followed prior art, it is easy to implement by following simple examples, and it side-steps the cat herding by always saying no to evolving its scope into anything more than it was 6 years ago. LAZ has some of these same properties (follow LAS, evolved slowly, and long stability), and we should be very careful about tinkering with it.
> What is more important? A well-written specification or one - or now two - reference implementations?
Ideally, I want a standard first, then one or two reference implementations, in a body that has a history of doing standards development, ratification, and promotion. Hobu's current efforts are focused on reading LAZ data, and while the laz-perf software can write byte-compatible LAZ, we aren't really doing so beyond some simple tests. There are no validation tests of LAZ beyond making sure LASzip can read/write it. There's no conformance, validation, or regression suite to protect against inadvertently screwing up. The only protection we've had thus far is taking great care. The advantage is both our implementations are open source software, and anyone can submit a patch to fix/change/improve it. This is its disadvantage too though, with ample possibility of all implementations being out of sync.
I think it would be wonderful for a CountryGovernmentDataAgency to fund you to write out a full document for how to make LAZ. This document would be a great starting point for standardization in the organization of your and CountryGovernmentDataAgency's choosing. In my opinion, it would be a worthwhile effort.
Howard