Train with images on node.js

[José Luis Alcalá Ramos]

Hi,

We are trying to train a model on node.js with images to recognize some sounds. And this is the code that we are using to convert the images to tensors -> 

const imageToBuffer = (path: string): any => {

    const img = fs.readFileSync(path);

    const canvImg = new canvas.Image;

    canvImg.src = img;

    return canvImg; 

}

const readImages = (): DsManager => {

    const dsManager = new DsManager(2);

    const directories = [{ folder: "sound-0", label: 0 }, { folder: "sound-1", label: 1 }] as DirectoryData[];

    const loadImg = (path:string,label:number) => {

        console.log(path, label);

        const img = tf.tidy(() => {

            const tfImage = tf.fromPixels(imageToBuffer(path));

            console.log(webcamImage);

            return tfImage.toFloat().div(tf.scalar(127)).sub(tf.scalar(1));

        });

        dsManager.addExample(img, label);

    }

    directories.forEach((dir: DirectoryData) => {

        fs.readdir(dir.folder, (err, files) => {

            const sortedFiles = files.sort((a: string, b: string) => {

                const aN = parseInt(a.replace('.png', ''), 10);

                const bN = parseInt(b.replace('.png', ''), 10);

                if (aN < bN) {

                    return -1;

                } else if (aN > bN) {

                    return 1;

                }

            });

            sortedFiles.forEach((file) => {+

                loadImg(`${dir.folder}/${file}`, dir.label);

            });

        });

    });

    return dsManager;

};

And we are getting the following error:

/Users/alcaljos/Dev/personal/tf-music/node_modules/@tensorflow/tfjs-node/dist/nodejs_kernel_backend.js:825

        throw new Error('Method not implemented.');

        ^

Error: Method not implemented.

    at NodeJSKernelBackend.fromPixels (/Users/torrealj/Dev/personal/tf-music/node_modules/@tensorflow/tfjs-node/dist/nodejs_kernel_backend.js:825:15)

    at Engine.fromPixels (/Users/torrealj/Dev/personal/tf-music/node_modules/@tensorflow/tfjs-core/dist/engine.js:296:29)

    at ArrayOps.fromPixels (/Users/torrealj/Dev/personal/tf-music/node_modules/@tensorflow/tfjs-core/dist/ops/array_ops.js:299:41)

    at /Users/torrealj/Dev/personal/tf-music/node_modules/@tensorflow/tfjs-core/dist/ops/operation.js:11:61

    at Object.Tracking.tidy (/Users/torrealj/Dev/personal/tf-music/node_modules/@tensorflow/tfjs-core/dist/tracking.js:36:22)

    at Object.descriptor.value [as fromPixels] (/Users/torrealj/Dev/personal/tf-music/node_modules/@tensorflow/tfjs-core/dist/ops/operation.js:11:26)

    at tf.tidy (/Users/torrealj/Dev/personal/tf-music/dist/main.js:33:36)

    at Object.Tracking.tidy (/Users/torrealj/Dev/personal/tf-music/node_modules/@tensorflow/tfjs-core/dist/tracking.js:36:22)

    at loadImg (/Users/torrealj/Dev/personal/tf-music/dist/main.js:30:24)

    at sortedFiles.forEach (/Users/torrealj/Dev/personal/tf-music/dist/main.js:63:18)

Any help will be appreciated!

Regards

[Nikhil Thorat]

Currently we don't have a fromPixels implementation in Node, but we are working on it: https://github.com/tensorflow/tfjs/issues/298

For now, you could load the images directly into an ArrayBuffer and pass it to a tensor constructor in Node.

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.
Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/24f90d11-bf67-4452-9843-944fdf1b5c13%40tensorflow.org.

[Tiago Vasconcelos]

Could you make an example on how to load the images to the ArrayBuffer and pass it to the tensor? There's already fromPixels() method available, but i'm not on the browser so i don't have canvas or Image()

[Roie Schwaber-Cohen]

You could use node-canvas. That said It'll be great to know how the tensor gets built one you do have it. I was unable to correctly resize (and reshape) my image based tensor and then use that tensor to query a pertained model (like mobilenet).

Does anyone know whats required in order to ensure the tensor is correctly shaped?

[Tiago Vasconcelos]

You can feed the image file to fs.readfile() and it returns a buffer... I'm working with this:

function parse(file) {

  let data = fs.readFileSync(file)

  return Array.from(data)

}

I'm facing an issue with 256x256 jpeg. I have 20000+ image files and js get's out of memory before building the array for the tensor. But i tested it with a small batch, like 20 images and tf.tensor(array_of_images).reshape([-1, 256, 256, 4]) and it worked!! Any idea how to handle data?! 

Should i reduce the images size? How does it affect the model accuraccy?