Neural network with 2 input in torch
72 views
Skip to first unread message
pedro...@gmail.com
Aug 24, 2016, 6:03:39 AM8/24/16
to torch7
Hi,
I'm creating an Eye tracking solution with neural network consisting on predict where the user is looking.
I'm trying to construct a neural network in Torch. I want to reproduce the attached scheme.
I have 2 correlated basis : one with faces other with just the position of the faces in the image. Each line represent one person and each image (face and positions) is labelised. I have 35 labels that correspond to 35 possibilities of gaze position.
I pass a convolution network to the faces.
I use parrallelTable to do some processing on the both table.
the problem is that when I want to train my model I can't use the fonction "trainer" because it takes just 1 argument and I have two.
I found an alternative is that I concat the table : each line represent the both images
I had image of 224*224 (faces) and images of 25*25 (positions) and now I have a line for each input of 224*224+25*25
then I can pass just one base to the train but I have to split it inside the neural network to do my convolution processing and others stuff.
Is there a native nn module that allows me to slice a tensor into a table with two elements? In order to have the correct input configuration for my parallel table.
help please
I'm creating an Eye tracking solution with neural network consisting on predict where the user is looking.
I'm trying to construct a neural network in Torch. I want to reproduce the attached scheme.
I have 2 correlated basis : one with faces other with just the position of the faces in the image. Each line represent one person and each image (face and positions) is labelised. I have 35 labels that correspond to 35 possibilities of gaze position.
I pass a convolution network to the faces.
I use parrallelTable to do some processing on the both table.
the problem is that when I want to train my model I can't use the fonction "trainer" because it takes just 1 argument and I have two.
I found an alternative is that I concat the table : each line represent the both images
I had image of 224*224 (faces) and images of 25*25 (positions) and now I have a line for each input of 224*224+25*25
then I can pass just one base to the train but I have to split it inside the neural network to do my convolution processing and others stuff.
Is there a native nn module that allows me to slice a tensor into a table with two elements? In order to have the correct input configuration for my parallel table.
help please
Wesley Chaney
Aug 25, 2016, 9:45:29 PM8/25/16
to torch7
I am having a hard time understanding exactly what you need.
Based on the diagram you are showing, it seems you want to compute 3 conv stacks, 2 share weights, and the third has a different size image than the two that share weights.
Assuming RGB images, you could setup as follows
input = {torch.Tensor(3,25,25),torch.Tensor(3,25,25),torch.Tensor(3,224,224)}
then use nn.ParallelTable which contains 3 nn.Sequential modules which each contain your convolutional processing pipelines (and you must explicitly share weights when you create them).
Another approach would be to to treat the data as
input = {torch.Tensor(2,3,25,25),torch.Tensor(1,3,224,224)} -- treating the two eyes as a batch to do the same computation on them
then use nn.ParallelTable with 2 Sequentials (one for each convolutional stack)
If this doesn't answer your question, let me know. This part is confusing to me "each input of 224*224+25*25"
Based on the diagram you are showing, it seems you want to compute 3 conv stacks, 2 share weights, and the third has a different size image than the two that share weights.
Assuming RGB images, you could setup as follows
input = {torch.Tensor(3,25,25),torch.Tensor(3,25,25),torch.Tensor(3,224,224)}
then use nn.ParallelTable which contains 3 nn.Sequential modules which each contain your convolutional processing pipelines (and you must explicitly share weights when you create them).
Another approach would be to to treat the data as
input = {torch.Tensor(2,3,25,25),torch.Tensor(1,3,224,224)} -- treating the two eyes as a batch to do the same computation on them
then use nn.ParallelTable with 2 Sequentials (one for each convolutional stack)
If this doesn't answer your question, let me know. This part is confusing to me "each input of 224*224+25*25"
Reply all
Reply to author
Forward
0 new messages