info
Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
CNNs: How are kernel/filter coefficients derived
33 views
Skip to first unread message
Eric
May 14, 2017, 3:24:07 AM5/14/17
to
Most of the docs on CNNs simply state that an image is convolved with
a bank of filters (or kernels), without detailing how the filters are
derived. Are there good explanations where the coefficients in a
given filter come from?
a bank of filters (or kernels), without detailing how the filters are
derived. Are there good explanations where the coefficients in a
given filter come from?
keghn feem
May 14, 2017, 9:42:35 AM5/14/17
to
Based on black box logic we give it data and it transforms it into
output data. We are not supposed to take a interest in what is going on
inside the box. From outside, the box have adjustment knobs that
tune the the out put to what we want. These knobs are the weights.
By chance on the first layers, the filters end up general being a edge
detector of vertical, diagonal, horizontal edges. And solid colors and corners
detectors of various positions.
Each filter kernel scans the whole image. So what one kernel knows, it can take its knowledge to another part of the image. Having a kernel anchored
in each location is too complex.
A friendly introduction to Convolutional Neural Networks and Image Recognition:
https://www.youtube.com/watch?v=2-Ol7ZB0MmU
How Deep Neural Networks Work:
https://www.youtube.com/watch?v=ILsA4nyG7I0
What Neural Networks See:
https://aiexperiments.withgoogle.com/what-neural-nets-see
bitter...@yahoo.ie
May 24, 2017, 12:54:23 AM5/24/17
to
They appear as a result of the training process (back propagation.) Obviously that is one reason why training times are so long. Of course you can reuse them if you want to train another similar net.
keghn feem
May 28, 2017, 4:36:19 PM5/28/17
to
Eric
May 28, 2017, 5:24:22 PM5/28/17
to
On Sun, 28 May 2017 13:36:17 -0700 (PDT), keghn feem
<kegh...@gmail.com> wrote:
>
>
>https://hackadaycom.files.wordpress.com/2017/05/deep-learning-diagram-themed.jpeg
>
Keghn, What is that showing, exactly? The 'filters' on the right are
hand-drawn, so is it illustrating the expected orientation of the real
filters as they are developed?
If so, this ties into my more recent question regarding training for
aerial photos, where the orientation can be rotated. I'd love to hear
your thoughts about that.
<kegh...@gmail.com> wrote:
>
>
>https://hackadaycom.files.wordpress.com/2017/05/deep-learning-diagram-themed.jpeg
>
Keghn, What is that showing, exactly? The 'filters' on the right are
hand-drawn, so is it illustrating the expected orientation of the real
filters as they are developed?
If so, this ties into my more recent question regarding training for
aerial photos, where the orientation can be rotated. I'd love to hear
your thoughts about that.
Eric
May 28, 2017, 5:30:02 PM5/28/17
to
On Tue, 23 May 2017 21:54:21 -0700 (PDT), bitter...@yahoo.ie
wrote:
>They appear as a result of the training process (back propagation.) Obviously that is one reason why training times are so long. Of course you can reuse them if you want to train another similar net.
Thanks. I was actually wondering how the backprop circuit works in
such a case. Unlike straight fully connected NN's, It has to traverse
pooling layers and such, then get directed into one of a multitude of
filters. It seemed a rather tenuous route to end up with such
geometrically oriented patterns in the first layer of filters. I have
seen much on this.
All the replies here have been interesting though, even if not
directly focused on that.
wrote:
>They appear as a result of the training process (back propagation.) Obviously that is one reason why training times are so long. Of course you can reuse them if you want to train another similar net.
such a case. Unlike straight fully connected NN's, It has to traverse
pooling layers and such, then get directed into one of a multitude of
filters. It seemed a rather tenuous route to end up with such
geometrically oriented patterns in the first layer of filters. I have
seen much on this.
All the replies here have been interesting though, even if not
directly focused on that.
keghn feem
May 28, 2017, 8:17:44 PM5/28/17
to
CNN have no problem dealing with orientation. Just give it the data and
make it match to a output label.
Just need to find images of aerial data base to train it on.
150 x 150 pixel in to the black box and 80 x 80 output pixels.
You get make the decision. That output pixel 44 x 55 is going to be
the top of a pine tree:) It is labeled top of pine tree by a link list.
Then from that out put pixel
back prop out all thing that down vote that output when the nn is shown a pine tree.
You can select a pixel to do cars at 40 degree for another output pixel
to do a truck 10 degrees.
If you want north or south orientation then gps data will have to entered
in at the beginning and may have to come in on it own filter.
To find what the a filter is trained to detect it is shown hand drawn
lines and see what activates it. it is possible for two or more filter
detecting the same thing. Since these little kernels are little NN and
not a hand crafted sobel edge detection algorithm.
Since they tiny NN 15 x 15 pixels with a output 5. Five slightly different
orientations of a line. All OR'd together for its output.
The activation of a filter are collected on a empty image in the
next layer. When a filter is not dead center over a line it will generate
a fractional output. Max pooling get rid of these:
https://www.quora.com/What-is-max-pooling-in-convolutional-neural-networks
Each feature is channeled out, by a filter, into its own red carpet path.
Then all of them are reconnected later on.
make it match to a output label.
Just need to find images of aerial data base to train it on.
150 x 150 pixel in to the black box and 80 x 80 output pixels.
You get make the decision. That output pixel 44 x 55 is going to be
the top of a pine tree:) It is labeled top of pine tree by a link list.
Then from that out put pixel
back prop out all thing that down vote that output when the nn is shown a pine tree.
You can select a pixel to do cars at 40 degree for another output pixel
to do a truck 10 degrees.
If you want north or south orientation then gps data will have to entered
in at the beginning and may have to come in on it own filter.
To find what the a filter is trained to detect it is shown hand drawn
lines and see what activates it. it is possible for two or more filter
detecting the same thing. Since these little kernels are little NN and
not a hand crafted sobel edge detection algorithm.
Since they tiny NN 15 x 15 pixels with a output 5. Five slightly different
orientations of a line. All OR'd together for its output.
The activation of a filter are collected on a empty image in the
next layer. When a filter is not dead center over a line it will generate
a fractional output. Max pooling get rid of these:
https://www.quora.com/What-is-max-pooling-in-convolutional-neural-networks
Each feature is channeled out, by a filter, into its own red carpet path.
Then all of them are reconnected later on.
0 new messages