OutOfMemory Error in OneDrive uploading file over a little large file

75 views
Skip to first unread message

ahs...@gmail.com

unread,
Jun 15, 2016, 4:45:50 AM6/15/16
to CodenameOne Discussions
Dear Sir/madam,

When i'm using connectionRequest to upload file to OneDrive, there is always out of memory error when the size of file is just a little large, say, over 60MB. Herewith attach a sample code i wrote.

/**
 * Your application code goes here
 */
package userclasses;

import com.codename1.io.BufferedOutputStream;
import com.codename1.system.NativeLookup;
import com.codename1.util.Base64;
import generated.StateMachineBase;
import com.codename1.ui.*;
import com.codename1.ui.events.*;
import com.codename1.ui.util.Resources;
import com.codename1.io.*;
import java.io.*;
import java.util.ArrayList;
import java.util.Date;
import java.util.Random;

public class StateMachine extends StateMachineBase {
    public StateMachine(String resFile) {
        super(resFile);
        // do not modify, write code in initVars and initialize class members there,
        // the constructor might be invoked too late due to race conditions that might occur
    }
   
    protected void initVars(Resources res) {
    }
    @Override
    protected void onMain_ButtonAction(Component c, ActionEvent event) {

            Display.getInstance().invokeAndBlock(new Runnable() {
                BackupOneDrive task = new BackupOneDrive();

                @Override
                public void run() {
//            new Thread(task).start();
                    task.run();

                }
            });
   
    }

    class BackupOneDrive extends Thread {
        public BackupOneDrive() {

        }

        public void run() {
            try {
               ConnectionRequest req = new ConnectionRequest() {


                    @Override
                    protected void buildRequestBody(OutputStream os) throws IOException {
                        System.out.println("Request Body");
                        byte[] b = new byte[157286400];
                        new Random().nextBytes(b);
                        InputStream inputStream = new ByteArrayInputStream(b);

                        synchronized (inputStream) {
                            BufferedOutputStream bos = new BufferedOutputStream(os);
                            try {
                                byte[] buffer = new byte[4096];
                                int length = 0;
                                while ((length = inputStream.read(buffer)) != -1) {
                                    System.out.println("length: " + buffer.toString());
                                    bos.write(buffer, 0, length);
                                    bos.flush();
                                }
                                inputStream.close();
                            } catch (Throwable e) {
                                e.printStackTrace();
                            } finally {
                                bos.close();
                            }
                        }

                    }
                };
                req.setDuplicateSupported(true);
                req.setSilentRetryCount(10);
                req.setTimeout(3000);
                NetworkManager.getInstance().setTimeout(3000);
                req.setReadResponseForErrors(false);
                req.setChunkedStreamingMode(4096);
                req.setPriority(ConnectionRequest.PRIORITY_CRITICAL);
                req.setPost(true);
                req.addResponseListener(new ActionListener() {
                    @Override
                    public void actionPerformed(ActionEvent evt) {
                        NetworkEvent e = (NetworkEvent) evt;
                        if (e.getError() != null) {
                            e.consume();
                            System.out.println("error: " + e.getError().getMessage());
                            return;
                        }
                    }
                });

                req.setUrl("https://apis.live.net/v5.0/{folder}/files/test.mp4?access_token={AccessToken}&downsize_photo_uploads=false");
                req.setHttpMethod("PUT");
                req.setContentType("");
                NetworkManager.getInstance().addToQueueAndWait(req);
                System.out.println("finish");
            } catch (Exception ex) {
                ex.printStackTrace();
            }
        }
    }

}

In that case, I just use a random byte about 150MB and the out of memory came out at about 60MB.
Moreover, the error happens in both simulator and real mobile.
Please advice and suggest us a solution.

Thanks and Regards,
Roanld

Shai Almog

unread,
Jun 16, 2016, 12:32:20 AM6/16/16
to CodenameOne Discussions, ahs...@gmail.com
Hi,
all output streams in Codename One are buffered so you should never wrap them in another buffered output stream.

You are allocating a single array of 150MB this will blow up on most devices... Writing it will usually trigger further allocations of similar size and will prevent the GC from doing anything about it since you are doing it in a loop. You need to work with smaller data sets.

ahs...@gmail.com

unread,
Jun 22, 2016, 6:37:49 AM6/22/16
to CodenameOne Discussions, ahs...@gmail.com
Hi,
It is because our design is required another buffered output stream, would you have any suggestion for that if really we need it?

Actually we have successfully uploaded in previous build about few months ago, and this problem is just happen recently. May you have any solution suggested for that size of file upload?

Thanks and Regards,
Ronald

Shai Almog

unread,
Jun 23, 2016, 1:10:51 AM6/23/16
to CodenameOne Discussions, ahs...@gmail.com
Hi,
I don't understand what design needs a buffered output stream.

On which OS did it used to work and stopped?

ahs...@gmail.com

unread,
Jun 24, 2016, 6:26:58 AM6/24/16
to CodenameOne Discussions, ahs...@gmail.com
Hi,
We are using Android to test this case.

When i add "addRequestHeader("Transfer-Encoding", "chunked");" in using gdrive upload, there is no out of memory issue anymore. However, it is not work for onedrive case, would you have any suggestion?

Thanks and Regards,
Ronald

Shai Almog

unread,
Jun 25, 2016, 2:13:11 AM6/25/16
to CodenameOne Discussions, ahs...@gmail.com

ahs...@gmail.com

unread,
Jun 26, 2016, 9:41:11 PM6/26/16
to CodenameOne Discussions, ahs...@gmail.com
It was fail.

Shai Almog

unread,
Jun 27, 2016, 12:13:25 AM6/27/16
to CodenameOne Discussions, ahs...@gmail.com
So if this isn't a regression then it's back to my original assertion that you are allocating a huge amount of RAM in a single block.

I suggest looking at MultipartRequest and it's source code to see how we handle large file uploads.

ahs...@gmail.com

unread,
Jun 27, 2016, 6:01:55 AM6/27/16
to CodenameOne Discussions, ahs...@gmail.com
Hi,
I'm trying to do multipart for onedrive and with the header "
addRequestHeader("Transfer-Encoding", "chunked");", however there is always error response "http 400  bad request", I have opened thread here http://stackoverflow.com/questions/38050903/one-drive-multipart-upload-error-http-400-bad-request

Please advice if you have any suggestion or would you have any example code for onedrive multipart?

Thanks and Regards,
Ronald

Reply all
Reply to author
Forward
0 new messages