tensorflow image processing

[Smitha Anil]

I want to compare the similarity between one reference window and all other windows taken from an image. My code is given below. 

Can anyone please help me evaluate the distance between 'ref' (reference window) and all other 10000 windows given by variable 'test'

# Read grayscale image from file.

Im = Image.open("cameraman.png") 

#Resize it to desired shape (h,w)

Im = Im.resize((100,100))

# expand dimensions to get the shape [ no of batches, height, width, channel]

Im = np.expand_dims(Im,axis=0)

Im = np.expand_dims(Im,axis=0)

x = tf.convert_to_tensor(Im)

x=tf.reshape(x,[1,100,100,1])  # this is the required image shape in a tensor

# Break one image into windows of 11x11 (overlapping)

wsize=11

ws=50  # Index of centre window (this window is reference window)

#Extract windows of 11 x 11 around each pixel

p1=tf.extract_image_patches(x,sizes=[1,wsize,wsize,1],strides=[1,1,1,1],rates=[1,1,1,1],padding="SAME")

patches_shape = tf.shape(p1)

test=tf.reshape(p1, [tf.reduce_prod(patches_shape[0:3]), 11, 11, ])  # returns [#window_patches, h, w, c] 

print(test.shape)  #test has shape [ 10000, 11,11]

ref=test[5000,]    # this is the reference window of shape [ 1, 11,11]

ref=tf.reshape(ref,[1,11,11])

print(im1.shape)

The following statement says size mismatch:

ssim1 = tf.image.ssim(ref, test, max_val=255, filter_size=11,filter_sigma=1.5, k1=0.01, k2=0.03)

Thank You

Smitha

[Dheeraj Rajaram Reddy]

The problem is that tf.image.ssim() works on single images only. A solution that works on multiple images will need some iterations over the tensor of patches. Here's my solution that's faster than iterating by using tf.map_fn with parallel iterations. 

import tensorflow as tf

x = tf.ones(shape=[1, 100, 100, 1])

patches = tf.image.extract_patches(x, 

        sizes=[1, 11, 11, 1], 

        strides=[1, 1, 1, 1], 

        rates=[1, 1, 1, 1], 

        padding="SAME")

patches_shape = tf.shape(patches)

patches = tf.reshape(patches, 

            [tf.reduce_prod(patches_shape[0: 3]), 11, 11, 1])

ref = patches[5000]

@tf.function

def _ssim(test_img):

    return tf.image.ssim(ref, test_img, max_val=255)

@tf.function

def batch_ssim(test_imgs):

    return tf.map_fn(_ssim, patches, parallel_iterations=4)

print(batch_ssim(patches))  # tensor of shape (10000,)

Running this on 10000 patches of size (11, 11) takes around ~5s on my laptop. This was written in the nightly version of TensorFlow, and it won't work in TensorFlow 1.x which I think is the version you're using. 

[Smitha Anil]

Thank you very much for the response . Yes I am using tensorflow 1 as you righlty pointed out. I am just a beginner in this tensorflow. I will check the version which you mentioned. And I will try the solution provided by you

Thank you once again