Re: parsing large json files

[Alin Parjolea]

Dude i haven't worked with big chunks of data miself just with small ones. So i think that you are right. Question : Do you create urself those json files or u already get them generated and u just want to decode?

On Wed, Jun 24, 2009 at 8:37 AM, mercutio <jbo...@googlemail.com> wrote:

hi there,

i am parsing quite large json files. at certain lengths the
json_decode function returns null.

I work on debian lenny which comes with php 5.2.6. this means that i
currently cannot use the json_last_error() function.

i came up with the idea to read the json file in chunks and pass them
one by one to json_decode.

After searching the web I did not find anything about this approach...
so, do you guys know of a best practice how to deal with very large
json data?

kind regards
mercutio

[Alin Parjolea]

And do you need to decode all that big data at once?

[Alin Parjolea]

if you could find a way to ... get the data maybe not entirely but based on depths. Or using Ajax i guess. I'm not profi in Ajax but i guess you can get the data nicely performing Ajax requests. and u dont need to get all the data at once from the db.

Does what i said make sense coz i just thought it and wrote.

On Wed, Jun 24, 2009 at 9:02 AM, Jens Bonerz <jbo...@googlemail.com> wrote:

the data represents a traversed directory structure from the user's
local file system.

this can contain many thousand folders and files.

the json data which contains the directory structure is compressed
before it is sent to the php application and is then stored in a
database.

the php application then makes this data available on demand and shall
visualize the json object as a folder structure on the website.

i am not quite sure how to do this yet, perhaps dynamically but this
would be the next step...

i guess I first need to be able to parse the entire json file into an
object or array and move on from there?

[Carlos Conti]

That’s definetely a nice approach…

Monitoring on a per-folder basis the data load status, it should be easy to determine the actions to be triggered on your tree component.

 

 

 

---