Store and retrieve points to/from binary file

[Marco Vettorello]

Hi all,

I'm quite new to WebGL and it's very interesting. I'm interested to display an high number of points from a file (from 10K to 1M or more), I've already developed the application, with vertex and  fragment shaders, the only thing I need is to load the data (now I currently randomly generate the points for test).

My problem is that I don't know how to store a binary representation of a Float32Array and how to retrieve it correctly from WebGL. Can someone give me some hints on how to create a file that represent a Float32Array and how is possible to load it on javascript?

I know that I can use a simple JSON and then convert it to float32Array, but it's not the best way to store float numbers.

Thanks

marco 

 

[Ben Adams]

If you aren't worried about endianness; then a Float32 is a regular 4-byte IEEE single precision number; so you can just pack them into a file and load with XMLHttpRequest.

Transfer with MIME type (Content-Type header) application/octet-stream

When loading ensure you set the responseType = 'arraybuffer';

For example:

var urlToFloatFile = 'http://....';

var request = new XMLHttpRequest();

request.open('GET', urlToFloatFile, true);

request.responseType = 'arraybuffer';

request.onload = function (msg) { 

    var yourFloatData = new Float32Array(this.response);

};

request.send();

Kind regards

@ben_a_adams

--
You received this message because you are subscribed to the Google Groups "WebGL Dev List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webgl-dev-lis...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Marco Vettorello]

Thanks Ben,

I successfully managed to store and retrieve a Float32Array using your code and your hints.

I used NodeJS to create the file writing the float in little-endians order, than I retrieve it using your code.

Here is the code for the file creation in NodeJS

    var fs = require('fs');
    var wstream = fs.createWriteStream('data.dat');

    var data = new Float32Array([1.1,2.2,3.3,4.4,5.5]);

    //prepare the length of the buffer to 4 bytes per float
    var buffer = new Buffer(data.length*4);


    for(var i = 0; i < data.length; i++){
        //write the float in Little-Endian and move the offset
        buffer.writeFloatLE(data[i], i*4);
    }

    wstream.write(buffer);
    wstream.end();

And your code for the file request on browser.

    var urlToFloatFile = 'data.dat';

    var request = new XMLHttpRequest();
    request.open('GET', urlToFloatFile, true);

    //specify the response type as arraybuffer

    request.responseType = 'arraybuffer';

    request.onload = function (msg) { 
        var yourFloatData = new Float32Array(this.response);

        console.log(yourFloatData);
    };
    request.send();

Now I can easily transport million of points in smaller files than having a big database some JSON file.

Thanks