Tensorflow gradient shape incompatible when using Convolutional Transpose Layer - neural-network

Question

I was having an issue when trying to create a convolution-deconvolution network. The original image dimensions are 565 * 584 and I'm trying to produce a segmentation of 565 * 584.
While I didn't have an issue before with my network with 1024*1024 images, I have been having some issues with these dimensions. I am getting this issue when computing the gradient:
segmentation_result.shape: (?, 565, 584, 1), targets.shape: (?, 565, 584, 1)
Process Process-1:
Traceback (most recent call last):
\Python\Python35\lib\site-packages\tensorflow\python\framework\tensor_shape.py", line 558, in merge_with
self.assert_is_compatible_with(other)
\Python\Python35\lib\site-packages\tensorflow\python\framework\tensor_shape.py", line 106, in assert_is_compatible_with
other))
ValueError: Dimensions 565 and 566 are not compatible
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
\Python\Python35\lib\multiprocessing\process.py", line 249, in _bootstrap
self.run()
\Python\Python35\lib\multiprocessing\process.py", line 93, in run
self._target(*self._args, **self._kwargs)
.py", line 418, in train
network = Network(net_id = count, weight=pos_weight)
.py", line 199, in __init__
self.train_op = tf.train.AdamOptimizer().minimize(self.cost)
\Python\Python35\lib\site-packages\tensorflow\python\training\optimizer.py", line 315, in minimize
grad_loss=grad_loss)
\Python\Python35\lib\site-packages\tensorflow\python\training\optimizer.py", line 386, in compute_gradients
colocate_gradients_with_ops=colocate_gradients_with_ops)
\Python\Python35\lib\site-packages\tensorflow\python\ops\gradients_impl.py", line 560, in gradients
in_grad.set_shape(t_in.get_shape())
\Python\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 443, in set_shape
self._shape = self._shape.merge_with(shape)
\Python\Python35\lib\site-packages\tensorflow\python\framework\tensor_shape.py", line 561, in merge_with
raise ValueError("Shapes %s and %s are not compatible" % (self, other))
ValueError: Shapes (?, 565, 584, 64) and (?, 566, 584, 64) are not compatible
The entire network has 10 convolutional layers and 10 deconvolutional layers. Each deconvolutional layer is a reversed version of the forward layer. Here is an example of the code to produce the convolutional layer:
def create_layer_reversed(self, input, prev_layer=None):
net_id = self.net_id
print(net_id)
with tf.variable_scope('conv', reuse=False):
W = tf.get_variable('W{}_{}_'.format(self.name[-3:], net_id),
shape=(self.kernel_size, self.kernel_size, self.input_shape[3], self.output_channels))
b = tf.Variable(tf.zeros([W.get_shape().as_list()[2]]))
output = tf.nn.conv2d_transpose(
input, W,
tf.stack([tf.shape(input)[0], self.input_shape[1], self.input_shape[2], self.input_shape[3]]),
strides=[1,1,1,1], padding='SAME')
Conv2d.layer_index += 1
output.set_shape([None, self.input_shape[1], self.input_shape[2], self.input_shape[3]])
output = lrelu(tf.add(tf.contrib.layers.batch_norm(output), b))
return output