Using a Trained Neural Network

[Rickesh Bedia]

Hi all, 

I am fairly new to using cortex so I have tried a simple neural network which I can successfully train.

This may be a bit of a silly question but how do I use the trained network?

Is there something, where you can call the network with a set of inputs and see what the outputs would be?

Something like: (network 0.0 0.0) => 0.75

This is my code:

(ns cortex-examples.core

  (:require [cortex.nn.execute :as execute]

                [cortex.nn.layers :as layers]

                [cortex.nn.network :as network]))

(def dataset

  [{:x [0.0 0.0] :y [0.0]}

   {:x [0.0 1.0] :y [1.0]}

   {:x [1.0 0.0] :y [1.0]}

   {:x [1.0 1.0] :y [0.0]}])

(def nn

  (network/linear-network

   [(layers/input 2 1 1 :id :x) 

    (layers/linear->tanh 10)

    (layers/linear 1 :id :y)]))

(defn train []

  (loop [network nn

         epoch 0]

    (if (> 30 epoch)

      (recur

        (:network (execute/train nn dataset

                                 :batch-size 4))

        (inc epoch))

      (clojure.pprint/pprint network))))

My resulting network after training is:

{:compute-graph

 {:nodes

  {:x

   {:output-channels 1,

    :output-height 1,

    :output-dimensions

    [{:channels 1, :height 1, :width 2, :stream :x}],

    :type :input,

    :output-width 2,

    :id :x,

    :output-size 2,

    :parents [],

    :input-dimensions [{:channels 1, :height 1, :width 2, :stream :x}],

    :input {:stream :x}},

   :linear-1

   {:output-size 10,

    :type :linear,

    :id :linear-1,

    :input-dimensions [{:channels 1, :height 1, :width 2, :id :x}],

    :weights {:buffer-id :linear-1-weights-1},

    :bias {:buffer-id :linear-1-bias-1}},

   :tanh-1

   {:type :tanh,

    :id :tanh-1,

    :input-dimensions

    [{:channels 1, :height 1, :width 10, :id :linear-1}],

    :output-dimensions [{:channels 1, :height 1, :width 10}]},

   :y

   {:output-size 1,

    :id :y,

    :type :linear,

    :input-dimensions

    [{:channels 1, :height 1, :width 10, :id :tanh-1}],

    :weights {:buffer-id :y-weights-1},

    :bias {:buffer-id :y-bias-1}},

   :mse-loss-1

   {:type :mse-loss,

    :labels {:type :stream, :stream :y},

    :output {:type :node-output, :node-id :y},

    :id :mse-loss-1,

    :input-dimensions [{:channels 1, :height 1, :width 1, :id :y}],

    :output-dimensions [{:channels 1, :height 1, :width 1}]}},

  :edges

  ([:x :linear-1] [:linear-1 :tanh-1] [:tanh-1 :y] [:y :mse-loss-1]),

  :buffers

  {:linear-1-weights-1

   {:buffer

    [[-0.7081067562103271, 0.7061067819595337]

     [-0.7081067562103271, 0.7061067819595337]

     [0.7081067562103271, -0.7061067819595337]

     [0.7081067562103271, -0.7061067819595337]

     [-0.7081067562103271, 0.7061067819595337]

     [0.7061067819595337, -0.7081067562103271]

     [0.7061067819595337, -0.7081067562103271]

     [-0.7061067819595337, 0.7081067562103271]

     [0.7081067562103271, -0.7061067819595337]

     [-0.7081067562103271, 0.7061067819595337]]},

   :linear-1-bias-1

   {:buffer

    [-9.999999310821295E-4, -9.999986505135894E-4,

     0.0010000000474974513, 0.0010000000474974513,

     -0.0010000000474974513, -0.0010000000474974513,

     -9.999999310821295E-4, 9.999999310821295E-4, 9.999999310821295E-4,

     -9.999999310821295E-4]},

   :y-weights-1

   {:buffer

    [-0.09929351508617401, -0.010217714123427868, 0.5990815758705139,

     0.2896946668624878, -0.46204477548599243, -0.4097350239753723,

     -0.1150742769241333, 0.1841706484556198, 0.25247427821159363,

     -0.22905582189559937]},

   :y-bias-1 {:buffer [0.0010000000474974513]}},

  :streams

  {:x {:channels 1, :height 1, :width 2},

   :y {:channels 1, :height 1, :width 1}}},

 :traversal

 {:forward

  ({:incoming [{:stream :x}],

    :id :linear-1,

    :outgoing [{:id :linear-1}]}

   {:incoming [{:id :linear-1}],

    :id :tanh-1,

    :outgoing [{:id :tanh-1}]}

   {:incoming [{:id :tanh-1}], :id :y, :outgoing [{:id :y}]}),

  :backward

  [{:incoming [{:id :y}], :id :y, :outgoing [{:id :tanh-1}]}

   {:incoming [{:id :tanh-1}],

    :id :tanh-1,

    :outgoing [{:id :linear-1}]}

   {:incoming [{:id :linear-1}],

    :id :linear-1,

    :outgoing [{:stream :x}]}],

  :buffers nil,

  :type :training}}

Any help is much appreciated,

Rickesh

[Harold]

Hi Rickesh,

The function you're looking for is `cortex.nn.execute/run`:

https://github.com/thinktopic/cortex/blob/master/src/cortex/nn/execute.clj#L471

Here's a simple example that uses it:

https://github.com/thinktopic/cortex/blob/master/experiment/test/cortex/experiment/logistic_test.clj#L46

Hope that helps,

-Harold

[Rickesh Bedia]

Hi Harold,

Thank you for your reply.

For my example, I tried doing:

(execute/run (train) [{:x [0.0 0.0]}])

But I am getting 

ExceptionInfo Network does not appear to contain a graph; keys should contain :compute-graph  clojure.core/ex-info (core.clj:4617)

How do I use the function to test what the output would be with an intial input of 0.0 0.0?

Thanks

Rickesh

[Harold]

Rikesh,

In the code you posted, the last line of the `train` function is `pprint`. `pprint` returns `nil`. If you pass `nil` as the first argument to `run` you'll get the error you mentioned.

You need to verify that the arguments you're passing to `run` are correct. Look again at the example I shared with you, it is similar to what you're trying to do.

Hope that helps,

-Harold

[Rickesh Bedia]

Hi Harold,

I have my input data 

(def dataset

  [{:x [0.0 0.0] :y [0.0]}

   {:x [0.0 1.0] :y [1.0]}

   {:x [1.0 0.0] :y [1.0]}

   {:x [1.0 1.0] :y [0.0]}])

And my network

(def nn

  (network/linear-network

   [(layers/input 2 1 1 :id :x) 

    (layers/linear->tanh 10)

    (layers/relu)

    (layers/dropout 0.9)

    (layers/linear 100)

    (layers/linear->softmax 100)

    (layers/linear 1 :id :y)]))

I train it using

(defn train-xor []

  (let [trained (train/train-n nn dataset dataset

                               :batch-size 4

                               :epoch-count 30

                               :simple-loss-print? true)]

    (execute/run nn dataset)

    (execute/run trained dataset)))

I want to test it using inputs 0.0 and 0.0

So I did

(execute/run (train-xor) [{:data [0.0 0.0]}])

But I still get the error:

ExceptionInfo Network does not appear to contain a graph; keys should contain :compute-graph  clojure.core/ex-info (core.clj:4617)

I don't know what I am doing wrong.

I tried following the example in https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fthinktopic%2Fcortex%2Fblob%2Fmaster%2Fexperiment%2Ftest%2Fcortex%2Fexperiment%2Flogistic_test.clj%23L46&sa=D&sntz=1&usg=AFQjCNEZuXJoGvBAQl5zGrePk5ZUj8dxhg

Thanks 

Rickesh

[Harold]

Hi Rickesh,

The last call in your `train-xor` function is to `execute/run`. 

`execute/run` returns results, not a network. So `train-xor` doesn't return a network. 

So, passing the result of `train-xor` to `execute/run` which you did `(execute/run (train-xor) [{:data [0.0 0.0]}])` fails.

To me, it doesn't make sense that `train-xor` is calling `execute/run` at all (assuming that what `train-xor` is doing is training, not running, the network).

Hope that helps,

-Harold