Transaction propagation

[Dave S]

I'm testing out using XADisk in a Java EE application deployed as a JCA connector.

I'm testing uploading and downloading files via a JAX-RS endpoint using streams since I'd like to support handling large files without having to intermediately convert to byte arrays.  On the upload side this is working just fine.  I use the InputStream received via JAX-RS to read bytes and write them to a XAFileOutputStreamWrapper.  This part works great and I can upload a 4GB text file without using a large amount of memory.

My problem is doing the reverse and attempting to pass the XAFileInputStreamWrapper out to the JAX-RS endpoint.  I have attempted just returning the InputStream(XAFileInputStreamWrapper) as the entity of the JAX-RS Response and I have attempted creating a StreamingOutput but I am getting a org.xadisk.filesystem.exceptions.NoTransactionAssociatedException.  At first the class handling all XADisk I/O was an EJB (which obviously has transactions) and the JAX-RS Resource class was just a normal POJO.  I have made the resource class an EJB and I have tried using the @TransactionAttribute(TransactionAttributeType.REQUIRES_NEW) annotation and it has not made a difference.  The JAX-RS resource calling the method on the EJB to access XADisk should all be happening in the same transaction so I'm not sure what the problem is here.  Could you possible shed some light on this?

I'm using current Java 1.8 runtime and Payara 164 as my application server.

This is not the full stack trace (if you want, I can generate, didn't want to bloat the forum) but it shows where it leaves Jersey and enters XADisk:
org.xadisk.filesystem.exceptions.NoTransactionAssociatedException: The method that was called can only be called with a transaction associated, butthere is no such transaction present.
    at org.xadisk.filesystem.NativeSession.checkIfCanContinue(NativeSession.java:1231)
    at org.xadisk.filesystem.virtual.NativeXAFileInputStream.checkIfCanContinue(NativeXAFileInputStream.java:320)
    at org.xadisk.filesystem.virtual.NativeXAFileInputStream.read(NativeXAFileInputStream.java:144)
    at org.xadisk.filesystem.virtual.NativeXAFileInputStream.read(NativeXAFileInputStream.java:137)
    at org.xadisk.additional.XAFileInputStreamWrapper.read(XAFileInputStreamWrapper.java:77)
    at org.glassfish.jersey.message.internal.ReaderWriter.writeTo(ReaderWriter.java:115)

[Nitin Verma]

Hi Dave,

From the method/line-numbers in stack-trace, and matching that with code, what I can see is that the transaction ("session") linked with the xadisk's inputstream has already been completed - completed either by commit or by rollback.
So, when we still try using this inputstream, we see that exception.

The reason of why the transaction, in which the inputstream was originally created, got completed now, needs to be figured out.

If your transactions are container-managed, you need to carefully check if the container is committing/rolling-back the managed-transaction at some place automatically; and then perhaps we are continuing to use that xadisk inputstream, after that transaction-completion.

Let me know if this could help.

Thanks,
Nitin

[david.s...@gmail.com]

I've been thinking about this a bit and trying different things and I'm not sure this will be possible.  What I'm trying to do is return an XADisk InputStream as the JAX-RS Response entity in order to not have to load the whole data into a byte[].  The JAX-RS runtime has to coordinate with the client to feed data via the stream then close it when it's done.  At that point how would I close the XADisk connection?  I have a simple maven based project I've been using to test out these concepts if you'd like to try it (attached) but I'm just not sure this is going to work the way I'd like, though it is interesting that it works fine for the writing TO an XADisk stream case.

After deployment of the application, a single access of this url will write a small text file named testFile:
http://localhost:8080/transactional-files/resources/files/write

Attemping to read the file with either of these will get the transaction error:
http://localhost:8080/transactional-files/resources/files/readStream/testFile
http://localhost:8080/transactional-files/resources/files/readStream2/testFile

but this works fine since it loads the full file contents into a byte[] and returns that:
http://localhost:8080/transactional-files/resources/files/read/testFile

Thanks,
Dave

[Nitin Verma]

Hi Dave,

From the stack-trace's lines:

    at org.xadisk.additional.XAFileInputStreamWrapper.read(XAFileInputStreamWrapper.java:77)

    at org.glassfish.jersey.message.internal.ReaderWriter.writeTo(ReaderWriter.java:115)

it appears that jersey is using the xadisk-inputstream at a point when the transaction linked to the stream is already completed (hence we see this exception, as expected).

Actually, the streams (input or output) of xadisk cannot be "exported/leaked" to any other code which would use these after the time of transaction completion. In this case,
we are exporting the inputstream, and by the time jersey uses it, the transaction is already complete.

For xadisk-outputstream in this example, we are  not really exporting anything out of the transaction: the transaction starts, xadisk-outputstream gets created, which is written using
a normal inputstream (non-xadisk), and the transaction completes. Then, that xadisk-outputstream is no longer used. So it worked for that case.

Thanks,
Nitin

[Dave S]

On Friday, January 6, 2017 at 9:34:06 AM UTC-5, Nitin Verma wrote:

Actually, the streams (input or output) of xadisk cannot be "exported/leaked" to any other code which would use these after the time of transaction completion. In this case,
we are exporting the inputstream, and by the time jersey uses it, the transaction is already complete.

OK, hadn't thought of it that way but I see why there's no issue there.  The Inputstream it's reading from in this case isn't transactional and all the transactional-related bits are enclosed in that one EJB method call.

 

For xadisk-outputstream in this example, we are  not really exporting anything out of the transaction: the transaction starts, xadisk-outputstream gets created, which is written using
a normal inputstream (non-xadisk), and the transaction completes. Then, that xadisk-outputstream is no longer used. So it worked for that case.

Again, made me think about it some more and made me determine that going lower-level was the only way to solve the problem so I wrote the following test and it works fine.  On the plus side I can close the XADisk connection this way as well.

@Transactional
@WebServlet("/download")
public class StreamingDownloadServlet extends HttpServlet {

    private static final Logger logger = Logger.getLogger(StreamingDownloadServlet.class.getName());

    @Inject
    FileService fileService;

    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        String fileName = req.getParameter("file");
        System.out.println("fileName = " + fileName);
        String cdString = String.format("attachment; filename=\"%s\"", fileName);
        resp.addHeader(HttpHeaders.CONTENT_DISPOSITION, cdString);
        try (OutputStream os = resp.getOutputStream()) {
            XADiskResponse xaResp = fileService.readStream2(fileName);
            byte[] buff = new byte[8096];
            int bytesRead;
            long totalBytesRead = 0;
            try (InputStream is = xaResp.getInputStream()) {
                while ((bytesRead = is.read(buff)) != -1) {
                    os.write(buff, 0, bytesRead);
                    totalBytesRead += bytesRead;
                }
            }
            logger.log(Level.INFO, "totalBytesRead: {0}", totalBytesRead);
            xaResp.closeConnection();
        }
    }
}

Thanks for looking into this and shedding some light on the why's and what-for's!  I should be able to use XADisk in my project now.

A couple things you might consider:
1) I don't know if you noticed but there's a typo in the exception message "butthere" instead of "but there"
 -  org.xadisk.filesystem.exceptions.NoTransactionAssociatedException: The method that was called can only be called with a transaction associated, butthere is no such transaction present.
2) Could you make the XAFileInputStreamWrapper, XAFileOutputStreamWrapper and XADiskConnection implement the java.io.Closeable interface?  They we could use try-with-resources construct!  I don't think you'd have to change the code other than just adding "implements Closeable" since they all have close methods.  You could potentially add this to any other classes in the project that have close methods.  I only mention the 3 I noticed.

Thanks for making such a useful open source project and supporting it.
Dave

[Nitin Verma]

Thanks Dave. I am making note of the suggestions you mentioned.

Nitin