I did more investigation on this and found issue is related to offsetting read from Start of image (SOI) marker in JPEG de-compression.
so if SOI is not offset then we are seeing graying image due to I believe bad input to de-compressor otherwise not.
Can you someone comment on SOI if it is mandatory in JPEG image.
Thanks.