IOException truncated read

[Danilo Câmara]

I am getting "java.io.IOException: truncated read" when scanning certain sizes with the latest jfreesane from source. Consider this code:

import java.io.IOException;
import java.net.InetAddress;

import au.com.southsky.jfreesane.SaneDevice;
import au.com.southsky.jfreesane.SaneException;
import au.com.southsky.jfreesane.SaneSession;

public class Test {

    public static void main(String[] args) throws IOException, SaneException {
        if (args.length < 2) return;

        double w = Double.parseDouble(args[0]);
        double h = Double.parseDouble(args[1]);

        SaneSession session = null;
        SaneDevice device = null;
        try {
            InetAddress address = InetAddress.getByName("localhost");
            session = SaneSession.withRemoteSane(address);

            device = session.getDevice("hpaio:/usb/Deskjet_4620_series?serial=CN582736YP47TN");

            device.open();

            device.getOption("source").setStringValue("Flatbed"); // [Flatbed], ADF
            device.getOption("mode").setStringValue("Color");     // [Lineart], Gray, Color
            device.getOption("resolution").setIntegerValue(200);  // 100, 200, 300, 600 (75)
            device.getOption("tl-x").setFixedValue(0.0);          // [0.0] -  215.90000915527344
            device.getOption("tl-y").setFixedValue(0.0);          // [0.0] -  297.01068115234375
            device.getOption("br-x").setFixedValue(w);            //  0.0  - [215.90000915527344]
            device.getOption("br-y").setFixedValue(h);            //  0.0  - [297.01068115234375]

            device.acquireImage();

        } finally {
            if (device != null) try { device.close(); } catch (IOException e) { e.printStackTrace(); }
            if (session != null) try { session.close(); } catch (IOException e) { e.printStackTrace(); }
        }
    }
}

This code runs fine with

$ java -cp .:./guava-18.0.jar Test 130.0 185.0

but if I try

$ java -cp .:./guava-18.0.jar Test 105.0 149.0

it throws:

Exception in thread "main" java.io.IOException: truncated read
    at au.com.southsky.jfreesane.FrameInputStream.readFrame(FrameInputStream.java:46)
    at au.com.southsky.jfreesane.SaneSession.acquireImage(SaneSession.java:173)
    at au.com.southsky.jfreesane.SaneDevice.acquireImage(SaneDevice.java:91)
    at Test.main(Test.java:34)

Attached is the saned output when the exception is thrown. Always reproducible. Same thing happens with another HP Scanner I have (hpaio:/net/Photosmart_Plus_B210_series?ip=192.168.1.66)

A appreciate if you have some information or any suggestions about what I should investigate in source code.

Thanks

[James Ring]

Doh! Does this happen with earlier versions of jfreesane (e.g. 0.91 or
0.9)? You can download the jars from
https://github.com/sjamesr/jfreesane/releases.

> --
> You received this message because you are subscribed to the Google Groups
> "jfreesane-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jfreesane-disc...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[James Ring]

Hmm, actually, can you try again without setting the tl-* and br-*
options? Maybe we're not correctly adjusting what we expect from sane
in the case that you ask for a smaller range.

If that works, can you set the options to (0, 0) and (215.9, 297.01)
and see if that works? I'll also try with my scanner when I get home.

Thanks,
James

[Danilo Câmara]

Same thing happens in jfreesane 0.91, 0.9, 0.8 and 0.7. Works fine without setting the tl-* and br-* or with range from (0, 0) to (215.9, 297.01).

Thanks

[James Ring]

Thank you Danilo, this helps a lot. I bet the problem is that sane is
sending the same pixels per line irrespective of the tl and br
settings and we expect to see exactly that many pixels.

Hopefully I'll be able to fix this sometime in the next few days, in
the meantime perhaps you can obtain the full page and do some
manipulation on the resulting BufferedImage.

[James Ring]

I couldn't reproduce the problem with the pixma backend. Can you try
one of the "more established" (less buggy) frontends like scanimage
with the same values for tl and br and see if that works?

[James Ring]

Also I added some additional logging. If you sync to the latest head,
can you send me the log output from this program? Make sure logging is
at FINE level or higher, you can use a technique like
https://github.com/sjamesr/jfreesane/commit/1f9c8ed2e6620e02302ffeea002d77ce92658ea8

[Danilo Câmara]

Scanimage as in this command line runs fine:

scanimage \
 --device-name=hpaio:/usb/Deskjet_4620_series?serial=CN582736YP47TN \
 --source=Flatbed \
 --mode=Color \
 --resolution=200 \
 -l 0.0 -t 0.0 -x 105.0 -y 149.0 > image.pnm

Scanimage options specific to this scanner are:

All options specific to device `hpaio:/usb/Deskjet_4620_series?serial=CN582736YP47TN':
  Scan mode:
    --mode Lineart|Gray|Color [Lineart]
        Selects the scan mode (e.g., lineart, monochrome, or color).
    --resolution 100|200|300|600dpi [75]
        Sets the resolution of the scanned image.
    --source Flatbed|ADF [Flatbed]
        Selects the scan source (such as a document-feeder).
  Advanced:
    --brightness 0..2000 [1000]
        Controls the brightness of the acquired image.
    --contrast 0..2000 [1000]
        Controls the contrast of the acquired image.
    --compression JPEG [JPEG]
        Selects the scanner compression method for faster scans, possibly at
        the expense of image quality.
    --jpeg-quality 0..100 [inactive]
        Sets the scanner JPEG compression factor. Larger numbers mean better
        compression, and smaller numbers mean better image quality.
  Geometry:
    -l 0..215.9mm [0]
        Top-left x position of scan area.
    -t 0..297.011mm [0]
        Top-left y position of scan area.
    -x 0..215.9mm [215.9]
        Width of scan-area.
    -y 0..297.011mm [297.011]
        Height of scan-area.

Attached is FINE level logging of the test program with the latest source o jfreesane and also the saned output.

Thanks

[James Ring]

Hey Danilo,

That was helpful, thank you. It looks to me like the scanner backend
(hpaio in this case) is returning one line too few. The sane logs also
show that an attempt is made to read some bytes, but 0 are read. I
suspect this is a problem in the backend. I've checked in
https://github.com/sjamesr/jfreesane/commit/85fb847c8379a4d9142ab0d1213e08dec3f14a16
to the truncated_image_hack branch, can you try your program with that
branch checked out?

Thanks for your help,
James

[Danilo Câmara]

I noticed its a cyclic problem. With the selected paramenters, it occurs every 16.002mm in height for 2.073mm regardless of image width. More specifically, it happens for H in 4.446mm to 6.519mm, 20.448mm to 22.521, 36.450 to 38.523, and so on.

As a smaller example, 10.0mm x 20.447mm runs fine:

$ java -cp .:./guava-18.0.jar Test 10.0 20.447
Sep 28, 2014 7:04:29 PM au.com.southsky.jfreesane.FrameInputStream readFrame
FINE: Reading frame: FrameInputStream{isBigEndian=false, parameters=SaneParameters{frameType=RGB, isLastFrame=true, bytesPerLine=234, pixelsPerLine=78, lineCount=160, depthPerPixel=8}}
Sep 28, 2014 7:04:29 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Read a record of 8188 bytes
Sep 28, 2014 7:04:29 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Read a record of 8188 bytes
Sep 28, 2014 7:04:29 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Read a record of 8188 bytes
Sep 28, 2014 7:04:29 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Read a record of 8188 bytes
Sep 28, 2014 7:04:29 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Read a record of 4688 bytes
Sep 28, 2014 7:04:29 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Reached end of records

but 10.0mm x 20.448mm doesn't:

$ java -cp .:./guava-18.0.jar Test 10.0 20.448
Sep 28, 2014 7:06:24 PM au.com.southsky.jfreesane.FrameInputStream readFrame
FINE: Reading frame: FrameInputStream{isBigEndian=false, parameters=SaneParameters{frameType=RGB, isLastFrame=true, bytesPerLine=234, pixelsPerLine=78, lineCount=161, depthPerPixel=8}}
Sep 28, 2014 7:06:24 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Read a record of 8188 bytes
Sep 28, 2014 7:06:24 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Read a record of 8188 bytes
Sep 28, 2014 7:06:24 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Read a record of 8188 bytes
Sep 28, 2014 7:06:24 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Read a record of 8188 bytes
Sep 28, 2014 7:06:24 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Read a record of 4688 bytes
Sep 28, 2014 7:06:24 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Reached end of records
Sep 28, 2014 7:06:24 PM au.com.southsky.jfreesane.FrameInputStream readFrame
WARNING: truncated read (got 37,440, expected 37,674 bytes)
Sep 28, 2014 7:06:24 PM au.com.southsky.jfreesane.FrameInputStream readFrame
WARNING: padded image with -234 null bytes
Exception in thread "main" java.awt.image.RasterFormatException: Data array too small (should be > 37673 )
    at sun.awt.image.ByteComponentRaster.verify(ByteComponentRaster.java:931)
    at sun.awt.image.ByteComponentRaster.<init>(ByteComponentRaster.java:201)
    at sun.awt.image.ByteInterleavedRaster.<init>(ByteInterleavedRaster.java:191)
    at sun.awt.image.ByteInterleavedRaster.<init>(ByteInterleavedRaster.java:113)
    at java.awt.image.Raster.createInterleavedRaster(Raster.java:648)
    at au.com.southsky.jfreesane.SaneImage.toBufferedImage(SaneImage.java:114)
    at au.com.southsky.jfreesane.SaneSession.acquireImage(SaneSession.java:184)
    at au.com.southsky.jfreesane.SaneDevice.acquireImage(SaneDevice.java:91)
    at Test.main(Test.java:45)
ABRT problem creation: 'success'

Tested in Fedora 20 and CentOS 7. Attached the logs for the original case with source from truncated_image_hack branch.

Thanks

[James Ring]

I bet somewhere there's some rounding problem in the backend driver.

[James Ring]

There's also a silly bug in my patch, can you try with
https://github.com/sjamesr/jfreesane/commit/5f9ad3efa026037582e69fcb260facd46c287630
?

I'll also try reproducing with the test and pixma backends. Good
detective work so far! :)

[James Ring]

It would also help to run scanimage with SANE_DEBUG_HPAIO=8, that
should cause a lot of output, something in there might give a hint as
to what's going on. I suspect JFreeSane might be incorrectly
calculating the bytes per line, see
http://www.sane-project.org/html/doc012.html section 4.3.8, there is
some text to the effect that some backends can use a "padded" image
format which I'm sure JFreeSane doesn't support.

[Danilo Câmara]

Instead of padding, what about assuming the image is one line shorter when there's not enough bytes returned? I agree it would be better to be able to do the exact same rounding.

Attached scanimage output for:

scanimage \
 --source=Flatbed \
 --mode=Color \
 --resolution=200 \
 -l 0.0 -t 0.0 -x 10.0 -y 20.448 > image20.448.pnm

[James Ring]

On Sun, Sep 28, 2014 at 5:38 PM, Danilo Câmara <dfca...@gmail.com> wrote:
> Instead of padding, what about assuming the image is one line shorter when
> there's not enough bytes returned? I agree it would be better to be able to
> do the exact same rounding.

Hopefully the patch I sent you should do this, it would be nice to see
if it works. I'm going to take your log and see if I can figure out
what's going on in the backend driver... I could also look at other
frontends (e.g. scanimage) to figure out what they do in this case.

Regards,
James

[Danilo Câmara]

The output with the latest patch:

$ java -cp .:./guava-18.0.jar Test 10.0 20.448

Sep 29, 2014 11:17:39 AM au.com.southsky.jfreesane.FrameInputStream readFrame

FINE: Reading frame: FrameInputStream{isBigEndian=false, parameters=SaneParameters{frameType=RGB, isLastFrame=true, bytesPerLine=234, pixelsPerLine=78, lineCount=161, depthPerPixel=8}}

Sep 29, 2014 11:17:39 AM au.com.southsky.jfreesane.FrameInputStream readRecord

FINE: Read a record of 8188 bytes

Sep 29, 2014 11:17:39 AM au.com.southsky.jfreesane.FrameInputStream readRecord

FINE: Read a record of 8188 bytes

Sep 29, 2014 11:17:39 AM au.com.southsky.jfreesane.FrameInputStream readRecord

FINE: Read a record of 8188 bytes

Sep 29, 2014 11:17:39 AM au.com.southsky.jfreesane.FrameInputStream readRecord

FINE: Read a record of 8188 bytes

Sep 29, 2014 11:17:39 AM au.com.southsky.jfreesane.FrameInputStream readRecord

FINE: Read a record of 4688 bytes

Sep 29, 2014 11:17:39 AM au.com.southsky.jfreesane.FrameInputStream readRecord

FINE: Reached end of records

Sep 29, 2014 11:17:39 AM au.com.southsky.jfreesane.FrameInputStream readFrame

WARNING: truncated read (got 37,440, expected 37,674 bytes)

Sep 29, 2014 11:17:39 AM au.com.southsky.jfreesane.FrameInputStream readFrame
WARNING: padded image with 117 null bytes

Exception in thread "main" java.awt.image.RasterFormatException: Data array too small (should be > 37673 )
    at sun.awt.image.ByteComponentRaster.verify(ByteComponentRaster.java:931)
    at sun.awt.image.ByteComponentRaster.<init>(ByteComponentRaster.java:201)
    at sun.awt.image.ByteInterleavedRaster.<init>(ByteInterleavedRaster.java:191)
    at sun.awt.image.ByteInterleavedRaster.<init>(ByteInterleavedRaster.java:113)
    at java.awt.image.Raster.createInterleavedRaster(Raster.java:648)
    at au.com.southsky.jfreesane.SaneImage.toBufferedImage(SaneImage.java:114)
    at au.com.southsky.jfreesane.SaneSession.acquireImage(SaneSession.java:184)
    at au.com.southsky.jfreesane.SaneDevice.acquireImage(SaneDevice.java:91)

    at Test.main(Test.java:50)
ABRT problem creation: 'success'

Thanks

[James Ring]

Can you try with this patch:
https://github.com/sjamesr/jfreesane/commit/bf02ceb76b110df728b63cc2d31838f5f8de26fe

[Danilo Câmara]

It's running now:

$ java -cp .:./guava-18.0.jar Test 10.0 20.448

Sep 29, 2014 12:30:42 PM au.com.southsky.jfreesane.FrameInputStream readFrame
FINE: Reading frame: FrameInputStream{isBigEndian=false, parameters=SaneParameters{frameType=RGB, isLastFrame=true, bytesPerLine=234, pixelsPerLine=78, lineCount=161, depthPerPixel=8}}
Sep 29, 2014 12:30:42 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Read a record of 8188 bytes
Sep 29, 2014 12:30:42 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Read a record of 8188 bytes
Sep 29, 2014 12:30:42 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Read a record of 8188 bytes
Sep 29, 2014 12:30:42 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Read a record of 8188 bytes
Sep 29, 2014 12:30:42 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Read a record of 4688 bytes
Sep 29, 2014 12:30:42 PM au.com.southsky.jfreesane.FrameInputStream readRecord
FINE: Reached end of records
Sep 29, 2014 12:30:42 PM au.com.southsky.jfreesane.FrameInputStream readFrame
WARNING: truncated read (got 37,440, expected 37,674 bytes)
Sep 29, 2014 12:30:42 PM au.com.southsky.jfreesane.FrameInputStream readFrame
WARNING: padded image with 234 null bytes

Thank you very much

[James Ring]

There should be one black line at the bottom of the image?

[Danilo Câmara]

Yes, there is, but the image from scanimage has too. I can open the scaniname PNM image in Geeqie image viewer, but ImageMagick complains about the image:

$ scanimage --source=Flatbed --mode=Color --resolution=200 -l 0.0 -t 0.0 -x 10.0 -y 20.448 > image20.448.pnm

$ identify image20.448.pnm
identify: unable to read image data `image20.448.pnm' @ error/pnm.c/ReadPNMImage/912.

$ gm identify image20.448.pnm
gm identify: Unexpected end-of-file (image20.448.pnm).
gm identify: Request did not return an image.

$ scanimage --source=Flatbed --mode=Color --resolution=200 --format=tiff -l 0.0 -t 0.0 -x 10.0 -y 20.448 > image20.448.tif

$ identify image20.448.tif
image20.448.tif TIFF 78x161 78x161+0+0 8-bit sRGB 37.7KB 0.000u 0:00.010
identify: Bogus "StripByteCounts" field, ignoring and calculating from imagelength. `TIFFReadDirectory' @ warning/tiff.c/TIFFWarnings/824.

$ gm identify image20.448.tif
image20.448.tif TIFF 78x161+0+0 DirectClass 8-bit 36.8K 0.000u 0:01

[James Ring]

That is interesting! So, do you think the workaround in JFreeSane
should be "if the read is truncated by exactly one line, pad the last
line with 0 bytes"? We could do it only if the backend type is hpaio?

[Danilo Câmara]

I agree with this approach for hpaio.