Running against the JSON maxLength wall.

[Kevin Ingwersen]

Hello there.

I have evolved drag0n to a point, where it needs to return large amount of data. I am speaking of AT LEAST 100kb. But it won't work. I googled around some, and realized that my string of data was litterally too long :o

How can i make current appJS behave, and just return the whone thing? I dont want to implement pagination in this place honestly ><

Regards, Ingwie

[Simon Horton]

Hi Kevin,

There should be no specific limitation on JSON, I have sent JSON packets of many megabytes into the browser (for example) and it works fine. Obviously there may be a limitation at some particular part of the appjs limitation or in some other code but there is nothing in JSON spec that limits it so much.

php has json_encode and json_decode and as I understand it those should also not be limited to any particular file length.

If you are using a GET and including the json as a parameter then you will hit a limit, but if that is the case switch to using POST and you are again unlimited in the amount of data to send. There is a php setting max_post_size or something similar that you can increase from 2 meg to a much larger size so apache does not cut off the post data. The only reason for this limit is to stop viruses and other bad software filling the server with post data.

/Simon

--
You received this message because you are subscribed to the Google Groups "appjs-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to appjs-dev+...@googlegroups.com.
To post to this group, send email to appj...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Kevin Ingwersen]

Hey Simon

Well what I am doing is:

- Clicking a link

- Link is connected to an event and triggers it

- Event tells the core to send the content of a page

- The content of the (php) page is generated from a command in Mac which is VERY long.

- PHP wants to parse that into some nice HTML.

- But actualy, the whole thing just freezes, and I know the command only takes 5 seconds - i let it run for 1ß minutes and it still did nothing.

Sometiems I get a jQuery error message telling me it had an uncaught exception with an unknown token of "102400".

Any clue?

[Simon Horton]

Hi Kevin,

It could be the cgi implementation is not working correctly, i.e. it could be there that it is breaking. 

If you are sending out a very large amount of data from php then be sure to include:-

ob_flush();

flush();

commands in the php each time around the loop, this will force php to begin sending the output (rather than leaving it in the output buffers), this is an easy way to increase perceived performance for the end user. 

If you open the developer tools and then look at the response, maybe the json is invalid at some point? It is not unknown for invalid json to break the browser especially if it is very large.

/Simon

[Kevin Ingwersen]

When i put the call into the front page, it loads just fine. I have all limits turned off since my PHP is running off an INI. But then I wonder why I am getting the odd error. I am not even getting any output from the site "as it loads". Ill have to tweak my nodejs router a little more it seems ._.

[Nicolas Embleton]

It seems to me that you could have potential problems:

1. Content-type set to HTML and not application/json (or appropriate) (When you say "PHP wants to parse that into some nice HTML."), could make some things troublesome like process time for example
   
2. CGI process length going over the limit? You have multiple timeouts depending on how you configured your system, can be CGI-based, can be proxy-based, or server-based (node.js here, so it "shouldn't" be). 

It feels there's a timeout that just drops the request and stops waiting (without notifying it failed). 

How did you configure your php timeouts? I followed a bit your project in the mailing list but don't know exactly where is the php, whether it's embedded or hosted somewhere. Either way it should obey to a specific timeout.

[Nicolas Embleton]

Hmm, I just read that:

"If you open the developer tools and then look at the response, maybe the json is invalid at some point? It is not unknown for invalid json to break the browser especially if it is very large."

And that:

"I have all limits turned off since my PHP is running off an INI"

I think because at some point one of the chain drops, the JSON is indeed invalid. What does the Chrome Inspector Tools says about the request? Does it sees anything coming in? Does it receives any information at all then breaks? Or does not receive anything?

On Monday, September 16, 2013 7:12:41 PM UTC+7, Kevin Ingwersen wrote: