nn.CAddTable() vs. nn.JoinTable()

[yongsk]

I am trying to solve XOR problem with skip-connection architecture.
So, here is what I tried.


nn.Sequential {
  [input -> (1) -> (2) -> (3) -> output]
  (1): nn.ConcatTable {
    input
      |`-> (1): nn.Sequential {
      |      [input -> (1) -> (2) -> (3) -> output]
      |      (1): nn.Linear(2 -> 1)
      |      (2): nn.Sigmoid
      |      (3): nn.Linear(1 -> 1)
      |    }
       `-> (2): nn.Sequential {
             [input -> (1) -> output]
             (1): nn.Linear(2 -> 1)
           }
       ... -> output
  }
  (2): nn.JoinTable
  (3): nn.Tanh
}

First, I tried to combine two parallel sequential modules, concatenated from input, using nn.JoinTable().
For sanity check, I forwarded torch.randn(2) to the defined network. It did not work.

So, I replaced it with nn,CAddTable(), and did the same thing for sanity check.
And now it looks like the network makes sense and works.

nn.Sequential {
  [input -> (1) -> (2) -> (3) -> output]
  (1): nn.ConcatTable {
    input
      |`-> (1): nn.Sequential {
      |      [input -> (1) -> (2) -> (3) -> output]
      |      (1): nn.Linear(2 -> 1)
      |      (2): nn.Sigmoid
      |      (3): nn.Linear(1 -> 1)
      |    }
       `-> (2): nn.Sequential {
             [input -> (1) -> output]
             (1): nn.Linear(2 -> 1)
           }
       ... -> output
  }
  (2): nn.CAddTable
  (3): nn.Tanh
}


Q) What makes the second code work, while the first not?
(Of course, I checked documents, but it is still confusing.)

Thank you.

[alban desmaison]

Hi,

nn.JoinTable will concatenate the tensors in the table along the given dimension. So in your case your output will be a tensor of size 2

nn.CAddTable will add the tensors in the table element wise. So in your case the output will be of size 1

I guess that your loss expects a tensor of size 1 as input given that your ground truth is of size 1, that is why the first one does not work while the second does.

[yongsk]

yes, I think you are correct.
The groundtruth in that network is 1 dimesion.
Thank you!!