Suppose the architecture is:
CNN --> MergeLayer --> CNN --> MergeLayer --> ...
Suppose the MergeLayer adjust the CNN output by adding a constant.
Is the CNN's weight update/back prop step based on the output of the following MergeLayer or the output of the CNN?
If the latter, how can one adjust this network so that the CNN's weight update is based on the output of the MergeLayer?