app.CreateWebServer return file

436 views
Skip to first unread message

Paul Norman

unread,
Oct 13, 2015, 4:54:48 AM10/13/15
to AndroidScript
Hi,

How with the WebServer do I get my servlet to return a file please?

I noticed from an error msg that it is based on Jetty, would documentation from that project help, or is the implementation different?

Thanks,

Paul

Paul Norman

unread,
Oct 13, 2015, 10:59:31 PM10/13/15
to AndroidScript
I have a list of possible files (not a natural directory listing) generated in DroidScript and displaying on a client device, and a servlet handling the request generated by any given client browser link click.

The servlet can determine from the request which member of the list was clicked, and from that the full filepath on its own device.

The selected file needs to be sent (with a header?) to the client device.

How do I make a response for that please?

This is what I do in onstart()

serv = app.CreateWebServer( 8099, "" ); 
serv.SetFolder( "" ); 
serv.AddServlet( "/list", OnServletList ); 
serv.AddServlet( "/file", OnServletFile );
serv.Start();    //Handle servlet requests. 
     
Here's how I so far successfully generate my list from a DrpidScript List using a private function getPlainList() and am preparing to send the file back.

I just can't see in the Docs how to set the response to be a header mimetype and file return.


function OnServletFile ( request, info ){

// pdfList.GetItemByIndex(j).title

    serv.SetResponse("<script>alert('File Not Available Yet'); window.history.back()</script>");
}

function OnServletList( request, info ) {
    
    var buildList = "";
    var buildHtml = "<html><head><title>" 
     + storeListCurrentNameGlobal
     + "</title></head>"
    + "<body>list</body></html>";
    
    makeList = getPlainList(",");
    makeList = makeList.split(",");

    for (var item = 0; item < makeList.length; item++){
        buildList += "<a href=\"/file?" + item + "\"</a>"
        + panStr.getFileName(makeList[item]) 
        + "</a><br>\n";
    }
    
    serv.SetResponse( buildHtml.replace("list",buildList) ); 

   // app.ShowPopup(  info.remoteAddress + " says: " + request.msg ); 
    
    }

Paul Norman

unread,
Oct 13, 2015, 11:37:17 PM10/13/15
to AndroidScript
P.S. purpose:— in a tutorial/teaching/workshop, if you are casting your screen, and WiFi-ed, people can see a file resource list in their device browsers relating to what's on the big screen, and they can immediately "download" anything they need as you talk.

I can change my "list" servlet to generate links pointing to a device PHP server, but it starts to get more difficult and complexto distribute and setup the App.

Steve Garman

unread,
Oct 14, 2015, 3:59:27 AM10/14/15
to AndroidScript
Paul,
I don't see any way of doing this without an extra level of complexity.

I can see no way of setting response headers in a servlet and even if you manage to set content-type to something like application/octet-stream, SetResponse is likely to choke on binary data.

It probably is not worth trying to persuade your attendees to download an app that uses xmlhttprequest to download the files as they won't all be using Android.

Your server could offer to email the resources to the attendees but without session cookies they would need to enter their address for each download.

If the resources are text you could just return them as the response and ask the attendees to copy/Paste them from their browser. Simple for some people but not everyone.

Same goes for asking them to LongTouch/right-click links.

Sorry. A response of this length should be more helpful than this. I have just put down my musings in case they trigger any more sensible ideas in you or someone else.

Dave Smart

unread,
Oct 14, 2015, 4:52:16 AM10/14/15
to androi...@googlegroups.com
Hi Paul,

As you probably know, if you put .html or .jpg or any other file in the folder which is pointed by the serv.SetFolder() method, then web server will serve those file up to the user's browser (or users App) in the normal way... and that is the mechanism that you should be using if possible.  

It's fine to use a servlet to generate a non-existent (virtual folder) of files but you need to generate list of 'real' http urls on the user's device end (remember the url underneath a piece of HTML text can look different to the link title ;)

Also, I would be tempted to write out the file list an HTML or json file to the server folder too and get the client end to download that file.... it will be better performance than the servlet method for multiple users (DS servlet requests have to become single threaded and cross the Javascript->Java bridge).

Hope that helps.

Regards
David

Paul Norman

unread,
Oct 14, 2015, 4:54:17 AM10/14/15
to AndroidScript
"It probably is not worth trying to persuade your attendees to download an app that uses xmlhttprequest to download the files as they won't all be using Android."

Its all achieved through their device's browser, that's the simplicity.

They don't need the app. Just the presenter does.

Browsers are just pointed on shared network to app's device IP and app's sever's port.

I'm past proof of concept phase.

No worries, just hoped DroidScript webserver had the ability to serve files from script, can achieve desired result in a combo way using php's built in server.

Thanks,

Paul

Dave Smart

unread,
Oct 14, 2015, 5:09:42 AM10/14/15
to AndroidScript
Just a further note on the servlet mechanism in DS...

DroidScript servlets were design primarily for a small number of client connections for making projects like personal weather stations etc and will not cope well with large numbers of simultaneous requests since each client call needs to be marshaled in a single thread across the Java->JavaScript bridge. 

To avoid this problem when serving many clients, you need to generate resources in the server folder and have them fetched from the client browser using AJAX calls or normal page navigation, which will be handled in a multi-threaded way.  

Also you might want to consider mixing in WebSockets (as we have done in our WiFi-JukeBox App on Google Play).  The advantage of WebSockets is that you can send data from your server app to the clients whenever you want too.... this means you can 'Push' notifications to all the clients from the server end too.... and WebSockets are very fast too :)

Regards
David

Paul Norman

unread,
Oct 14, 2015, 5:13:26 AM10/14/15
to AndroidScript
Thanks Dave,

The files would be scattered across sdisks. Only united artifically in lists.

Toying with this general mechanism (to keep stream ASCII/utf8-sh) as my response format to a request, another overloaded link...

Courtesey...
http://stackoverflow.com/questions/5889345/is-it-safe-to-use-a-href-data-to-display-images

Made appropriate to the file type —

<a href="data:image/XYZ;base64,...(blah blah base64-encoded file goes here)..."
alt="File embedded using base64 encoding!">Click to Save</a>

Does that look feasible?

Thanks
Paul

Dave Smart

unread,
Oct 14, 2015, 5:21:37 AM10/14/15
to AndroidScript
I think it should work... try it and let us know :)

Paul Norman

unread,
Oct 14, 2015, 7:05:18 AM10/14/15
to AndroidScript
Will do Dave.

Thanks for all the info! I'm sure the Backgrounders you've just given will really help many of us make better and more effective and atuned apps Dave.

The sockets push info is great thanks.

Jetty states itself that it has now developed along the lines not of a traditional web server as such, but more as an inter app type setup.

I'm needing to use this in small groups - but will explore your advice in case someone needs it for larger situations.

I could do well to look at temporarily marshalling the list's referenced files in a traditional static website approach, generating pages that make Ajax calls from the client browsers as you recommend, especially for larger group sessions, but I'll explore the base64 approach first for its relative simplicity.

I don't know yet, but modern device browsers may accept a raw string —

"data:type/XYZ;base64,...(blah blah base64-encoded File goes here)..."


... and just throw up the save dialogue or just start saving to a default location. Will explore and see, and report back when I can.

Paul

Paul Norman

unread,
Oct 14, 2015, 10:49:05 AM10/14/15
to AndroidScript
Just checking please.

For reading things like PDF files off disk does app.ReadFile() have any additional encoding options as well as Encoding-options: windows-1252?

Otherwise I think I won't be able to get non-text files like .pdf or .7z etc in for btoa() or http://phpjs.org/functions/base64_encode/ to work with.

Paul

Dave Smart

unread,
Oct 14, 2015, 11:17:47 AM10/14/15
to AndroidScript
Well app.ReadFile() is currently written to cope with text based file only (unfortunately), but you can use the following mode strings:-

ISO-8859-1
  • US-ASCII
  • UTF-16
  • UTF-16BE
  • UTF-16LE
  • UTF-8

Paul Norman

unread,
Oct 14, 2015, 9:53:19 PM10/14/15
to AndroidScript
Thanks Dave, ran it through using a PDF file, knowing the binary (images fonts etc) would be mangled.

But Pdfs have certain text based instructions in them.

So my saved PDF has the correct number of pages albeit blank of course.

(By the way, http://phpjs.org has a wealth of JS ports of PHP functions I've found to be very reliable.)

Ill substitute the use of http://phpjs.org/functions/base64_encode/

with a DroidScript call to local android PHP built-in server app to get my binaries' base64 strings on demand mean while.

here's the basic test...

<code>

function OnServletFileWanted ( request, info ){

// extract list index j from request string
// pdfEtcList.GetItemByIndex(j).title

// https://developer.mozilla.org/en/docs/Web/API/WindowBase64/Base64_encoding_and_decoding

var hold = app.ReadFile(myFile); //windows-1252

// Wanted to use binary file with following but can't get binary meanwhile

// https://developer.mozilla.org/en-US/docs/Web/API/Blob

//var oMyBlob = new Blob(holdA, {type : 'application/pdf'}); // the blob

// var reader = new FileReader();

// etc... reader.readAsDataURL(oMyBlob); /

// using phpjs instead for crude test
// http://phpjs.org/functions/base64_encode/

// until DroudScript might read binary, replace following and get base64 from local android php server app

var dataString = base64_encode(hold);

dataString = "<a download=\"EmbedLaTeXfonts.pdf\" href=\"data:application/pdf;base64,"

              + dataString + "\">Click to Save</a>";

serv.SetResponse(dataString);

}

</code>

As I said my file downloaded from Chrome and saved in the default location as asked for: "EmbedLaTeXfonts.pdf" correct number of pages - but blank.

I've put a request on the wish list for binary reading capability.

Paul

Chris

unread,
Oct 15, 2015, 1:05:08 AM10/15/15
to AndroidScript
Paul, just so you know, you can read binary files now, just not with app.ReadFile, this is designed for text.

Here is the info, with examples:
http://wiki.droidscript.me.uk/doku.php?id=sample_code:random_file_acces

Steve Garman

unread,
Oct 15, 2015, 1:08:21 AM10/15/15
to AndroidScript
Coincidentally, I've just added a request for a base64 mode to ReadData

Paul Norman

unread,
Oct 15, 2015, 5:28:22 AM10/15/15
to AndroidScript
Thanks for that, DroidScript truly has many hidden nuggets of gold!

I am looking at something like the following how does all this look please?

Using...

// http://stackoverflow.com/questions/23190056/hex-to-base64-converter-for-javascript ...

function hexToBase64(str) {
  return btoa(String.fromCharCode.apply(null,
    str.replace(/\r|\n/g, "").replace(/([\da-fA-F]{2}) ?/g, "0x$1 ").replace(/ +$/, "").split(" "))
  );
}

// (Do I need to tell CreateFile() its opening binary? "rb" doesn't work, left it as simply "r")..


function OnServletFileWanted ( request, info ){

// To do checkout more fully ... - https://developer.mozilla.org/en/docs/Web/API/WindowBase64/Base64_encoding_and_decoding

// test file...

myFile = "/storage/sdcard1/Docs/e-Books/ebooks latex/EmbedLaTeXfonts.pdf";

var file = app.CreateFile( myFile, "r" );       
var len = file.GetLength();

file.Seek(0); // redundant on first use?

// can this be all in one, or is it required to be incrementally read?...

var hold = file.ReadData(len, "Hex");

file.Close();

var dataString = hexToBase64(hold);

dataString = "<a download=\"EmbedLaTeXfonts.pdf\" href=\"data:application/pdf;base64,"

              + dataString + "\">click to save</a>";

serv.SetResponse(dataString);

}

Steve Garman

unread,
Oct 15, 2015, 6:21:42 AM10/15/15
to AndroidScript
The hex to base64 is promising but is not quite going to cut it.

The regexes expect a pure hex string with (possibly) \r and \n to be removed.

What ReadData is returning is a comma-separated string of hex values.

Removing the commas will not be enough as the (2 digit) values do not have a leading zeroes.

It should be reasonably easy to do in a loop but whether this is getting to be too much work is a question.

If you need help building the loop I'm happy to write one for you but I suspect it's well within your abilities.

Steve Garman

unread,
Oct 15, 2015, 7:00:11 AM10/15/15
to AndroidScript
I believe this will produce the input required by hexToBase64

    var i, len, s="", arr=[];
    var file = app.CreateFile( "/sdcard/img1.jpg", "r" ); 
    var len = file.GetLength();  
    var whole = file.ReadData( len,"hex" ); 
    whole = 0+ whole.replace(/,/g,",0" );
    arr=whole.split(",");
    len=arr.length;
    for(i=0;i<len;i++)
    {
         s += arr[i].slice(-2);
    }
    app.ShowPopup(s);

Paul Norman

unread,
Oct 15, 2015, 7:14:21 AM10/15/15
to AndroidScript
Thanks very much for that Steve,

Will be useful for constructing HTML emails too.

I hadn't explored what ReadData() "Hex" was returning yet (I think I would be expecting a stream representation, but something lending itself to an array makes sense).

Just want to look through a couple of things for any nuances in all this to make sure the browser can accurately decode it with no embedded hidden flaws for the downloaded "file" on disk, like what are MDN saying about it...

https://www.google.com/url?q=https%3A%2F%2Fdeveloper.mozilla.org%2Fen%2Fdocs%2FWeb%2FAPI%2FWindowBase64%2FBase64_encoding_and_decoding

And glance at these...

http://phpjs.org/functions/base64_encode/

http://phpjs.org/functions/pack/

Ported to JavaScript PHP type functions do this for us—

var My64baseString = base64_encode(pack("H*", myHexRepresentation));

Just want to be sure what is expected for myHexRepresentation - I've seen a few things in forums that have alerted me.

Thanks again to Chris too.

Paul

Steve Garman

unread,
Oct 15, 2015, 7:27:35 AM10/15/15
to AndroidScript
In php pack,
"H*" (capital h) expects a series of 2 character strings (high nibble first) which is what my loop produces.

http://php.net/manual/en/function.pack.php

Paul Norman

unread,
Oct 15, 2015, 11:10:17 PM10/15/15
to AndroidScript
Thanks Steve!

I'll have a look at that when I next can.

Down here in the antipodes we're 12 hours ahead, so sorry for the dealay in answering.

Paul

Paul Norman

unread,
Oct 16, 2015, 4:13:34 AM10/16/15
to AndroidScript
Dear Steve,

The results were as nice and clean as your code!

My test 182kb .PDF comes out very well as a "download" to sdisk.

Only problem was when I tried to process a small resized 271kb JPEG I got a message "Script Error: Maximum call stack size exceeded" and of course a corrupt JPEG downloaded.

This line was in red in the IDE...

return btoa(String.fromCharCode.apply(null,


Thats a pretty small sized JPEG really, a modern device takes them in Mbs now.

May be we need to chunk this some how or it really needs to be done deeper down in DroidScript's innards?

For reference current code below...

Paul

function hexToBase64(str) { 
  return btoa(String.fromCharCode.apply(null, 
    str.replace(/\r|\n/g, "").replace(/([\da-fA-F]{2}) ?/g, "0x$1 ").replace(/ +$/, "").split(" ")) 
  ); 



function binaryBase64(thisFile){


// courtesey Steven Garman...
var i, len, s="", arr=[]; 
    var file = app.CreateFile( thisFile, "r" );  
    var len = file.GetLength();   
    var whole = file.ReadData( len,"Hex" );  
        file.Close();

    whole = 0+ whole.replace(/,/g,",0" ); 
    arr=whole.split(","); 
    len=arr.length; 
    for(i=0;i<len;i++) 
    { 
         s += arr[i].slice(-2); 
    } 

 
return   hexToBase64( s);
                       
}


function OnServletFile ( request, info ){

// pdfList.GetItemByIndex(j).title

// more notes - https://developer.mozilla.org/en/docs/Web/API/WindowBase64/Base64_encoding_and_decoding

// myFile = "/storage/sdcard1/Docs/e-Books/ebooks latex/EmbedLaTeXfonts.pdf";

myFile = "/storage/sdcard1/DCIM/IMG_20130313_070544.jpg";

var dataString = binaryBase64(myFile);

// http://stackoverflow.com/questions/11415665/save-base64-string-as-pdf-at-client-side-with-javascript
/*
<a download=pdfTitle href=pdfData title='Download pdf document' />
where pdfData is your base64 encoded pdf like "data:application/pdf;base64,

*/

// dataString = "<a download=\"EmbedLaTeXfonts.pdf\" href=\"data:application/pdf;base64,"

dataString = "<a download=\"kangaroo.jpg.\" href=\"data:image/jpeg;base64,"

              + dataString + "\">click to save</a>";

// alert(dataString);

serv.SetResponse(dataString);

}

Steve Garman

unread,
Oct 16, 2015, 5:05:40 AM10/16/15
to AndroidScript
Chunking reads is a piece of cake
http://wiki.droidscript.me.uk/doku.php?id=sample_code:hexview

Unfortunately, passing chunks to btoa is probably not going to result in base64 chunks you can just concatenate together, though if you choose the chunksize carefully, it may not be impossible to work around.

Whilst this is an interesting project, you want to use it in the real world. By now I would be looking back to plan BB or on to C etc.

The php server plan looks viable.

Alternatively, perhaps you can use a purely jetty approach if you are able to store base64 versions of your resources on the sdisks.

Dave Smart

unread,
Oct 16, 2015, 5:23:55 AM10/16/15
to androi...@googlegroups.com
I don't mean to be rude Guys... I'm sure you are learning stuff on the way, but with all the time you have spent on this problem so far, one of you could have written a little DroidScript plugin by now to read and write files as base64 data.

(and then you could give me the source code and I could bang it into DS when I have the time)

Just a thought :)

Paul Norman

unread,
Oct 16, 2015, 5:31:15 AM10/16/15
to AndroidScript
Thanks Steve I'll be really fine with either alternatative - making and storing on demand (testing for existance first and bearing versioning in mind) is good, just asking PHP server for passthru back to client browser is attractive - no worries.

Persevered with this seeing a much wider general usefulness for it in DroidScript.

On the same general theme, can we take our binary hex version through to making a blob that html5 can work with?

For security reasons, I assume we are blocked out from directly loading from sdcard the files needed for the underlying browser engine - e.g. for FileReader

https://developer.mozilla.org/en-US/docs/Web/API/FileReader

... there's a tremendous amount - already built right in (not just base64!) - we can do if we can get properly formed binary blobs acceptible to the browser engine!

Paul

Reply all
Reply to author
Forward
0 new messages