sqlrow to json again

[vince]

http://groups.google.com/group/web2py/browse_thread/thread/8e3e33879974663f/64f76c562366aec4?lnk=gst&q=json#64f76c562366aec4

will it be added or we'll have to wait? i think it's hard to avoid all
dependency contrib / thirdparty package. as long as it's include i
think that's fine.


-vince

[Timothy Farrell]

I've been using this in my setup (my setup has a few customizations) since the thread linked below.  Just to say that it works!

-tim

-- 
Timothy Farrell <tfar...@swgen.com>
Computer Guy
Statewide General Insurance Agency (www.swgen.com)

[Timothy Farrell]

Massimo, would it help if I made a bzr branch?

[mdipierro]

OK. it is in trunk but slightly different. The representation is now
independent on the database backend.

Massimo

On Dec 29, 8:04 am, Timothy Farrell <tfarr...@swgen.com> wrote:
> Massimo, would it help if I made a bzr branch?
> Timothy Farrell wrote:I've been using this in my setup (my setup has a few customizations) since the thread linked below. Just to say that it works!
> -tim

> vince wrote:http://groups.google.com/group/web2py/browse_thread/thread/8e3e33879974663f/64f76c562366aec4?lnk=gst&q=json#64f76c562366aec4will it be added or we'll have to wait? i think it's hard to avoid all dependency contrib / thirdparty package. as long as it's include i think that's fine. -vince-- Timothy Farrell<tfar...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)-- Timothy Farrell<tfar...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)

[Timothy Farrell]

You might want to switch back to list comprehensions.  Granted the code is uglier but there are two big reasons.

1) Speed. See http://www.daniweb.com/forums/thread32869.html for more info.
    You could wrap the inner loop inner workings in a function and still reap massive benefits
    by not having to call list.append() n^2 times.
2) It's recommended for building lists in Py3k.

The results of the above linked test are:

[0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58]
[0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58]

Profile of evenList1(), check the time consumed by the many append() calls:
(ignore time consumed by the profiler itself)

         50005 function calls in 0.174 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    50000    0.066    0.000    0.066    0.000 :0(append)
        1    0.002    0.002    0.002    0.002 :0(range)
        1    0.005    0.005    0.005    0.005 :0(setprofile)
        1    0.001    0.001    0.168    0.168 <string>:1(<module>)
        1    0.100    0.100    0.168    0.168 Script1.py:5(evenList1)
        1    0.000    0.000    0.174    0.174 profile:0(evenList1())
        0    0.000             0.000          profile:0(profiler)


Profile of evenList2():
(ignore time consumed by the profiler itself)

         5 function calls in 0.018 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.002    0.002    0.002    0.002 :0(range)
        1    0.000    0.000    0.000    0.000 :0(setprofile)
        1    0.001    0.001    0.018    0.018 <string>:1(<module>)
        1    0.015    0.015    0.017    0.017 Script1.py:15(evenList2)
        1    0.000    0.000    0.018    0.018 profile:0(evenList2())
        0    0.000             0.000          profile:0(profiler)

[mdipierro]

where? Can you send me a patch?

Massimo

On Dec 29, 9:01 am, Timothy Farrell <tfarr...@swgen.com> wrote:
> You might want to switch back to list comprehensions. Granted the code is uglier but there are two big reasons.

> 1) Speed. Seehttp://www.daniweb.com/forums/thread32869.htmlfor more info.

> mdipierro wrote:OK. it is in trunk but slightly different. The representation is now independent on the database backend. Massimo On Dec 29, 8:04 am, Timothy Farrell<tfarr...@swgen.com>wrote:Massimo, would it help if I made a bzr branch? Timothy Farrell wrote:I've been using this in my setup (my setup has a few customizations) since the thread linked below. Just to say that it works! -tim vince wrote:http://groups.google.com/group/web2py/browse_thread/thread/8e3e33879974663f/64f76c562366aec4?lnk=gst&q=json#64f76c562366aec4willit be added or we'll have to wait? i think it's hard to avoid all dependency contrib / thirdparty package. as long as it's include i think that's fine. -vince-- Timothy Farrell<tfar...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)-- Timothy Farrell<tfar...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)-- Timothy Farrell<tfar...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)

[Timothy Farrell]

Yes, but before I do that, I want to clarify something.

My first implementation results in a JSON list of objects like so (roughly):

[ {field1:'field1val',field2:'field2val'}, {field1:'field1val',field2:'field2val'} ]

Your reimplementation results in a JSON list of lists where the first list is the field (column) names like so:

[ ['field1', 'field2'], ['field1val', 'field2val'], ['field1val', 'field2val'] ]

My implementation is more useful for programming in Javascript.  With your implementation (while more efficient on the wire), requires that we have to unpack the data in client-side Javascript which adds an extra step for the developer and, if the user has either IE and/or a slow computer this could add an undesirable delay for large datasets (which was the whole point of my using list comprehensions in the first place).

So can I send you a patch redone with my method (JSON list of objects)?

-tim

[mdipierro]

Sorry. I must have got confused. I do not think I have seen your
implementation than.
Please email it to me.

Massimo

On Dec 29, 9:22 am, Timothy Farrell <tfarr...@swgen.com> wrote:
> Yes, but before I do that, I want to clarify something.
> My first implementation results in a JSON list of objects like so (roughly):[ {field1:'field1val',field2:'field2val'}, {field1:'field1val',field2:'field2val'} ]
> Your reimplementation results in a JSON list of lists where the first list is the field (column) names like so:
> [ ['field1', 'field2'], ['field1val', 'field2val'], ['field1val', 'field2val'] ]
> My implementation is more useful for programming in Javascript. With your implementation (while more efficient on the wire), requires that we have to unpack the data in client-side Javascript which adds an extra step for the developer and, if the user has either IE and/or a slow computer this could add an undesirable delay for large datasets (which was the whole point of my using list comprehensions in the first place).
> So can I send you a patch redone with my method (JSON list of objects)?
> -tim

> mdipierro wrote:where? Can you send me a patch? Massimo On Dec 29, 9:01 am, Timothy Farrell<tfarr...@swgen.com>wrote:You might want to switch back to list comprehensions. Granted the code is uglier but there are two big reasons. 1) Speed. Seehttp://www.daniweb.com/forums/thread32869.htmlfor more info. You could wrap the inner loop inner workings in a function and still reap massive benefits by not having to call list.append() n^2 times. 2) It's recommended for building lists in Py3k. The results of the above linked test are:[0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58] [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58] Profile of evenList1(), check the time consumed by the many append() calls: (ignore time consumed by the profiler itself) 50005 function calls in 0.174 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 50000 0.066 0.000 0.066 0.000 :0(append) 1 0.002 0.002 0.002 0.002 :0(range) 1 0.005 0.005 0.005 0.005 :0(setprofile) 1 0.001 0.001 0.168 0.168 <string>:1(<module>) 1 0.100 0.100 0.168 0.168 Script1.py:5(evenList1) 1 0.000 0.000 0.174 0.174 profile:0(evenList1()) 0 0.000 0.000 profile:0(profiler) Profile of evenList2(): (ignore time consumed by the profiler itself) 5 function calls in 0.018 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.002 0.002 0.002 0.002 :0(range) 1 0.000 0.000 0.000 0.000 :0(setprofile) 1 0.001 0.001 0.018 0.018 <string>:1(<module>) 1 0.015 0.015 0.017 0.017 Script1.py:15(evenList2) 1 0.000 0.000 0.018 0.018 profile:0(evenList2()) 0 0.000 0.000 profile:0(profiler) mdipierro wrote:OK. it is in trunk but slightly different. The representation is now independent on the database backend. Massimo On Dec 29, 8:04 am, Timothy Farrell<tfarr...@swgen.com>wrote:Massimo, would it help if I made a bzr branch? Timothy Farrell wrote:I've been using this in my setup (my setup has a few customizations) since the thread linked below. Just to say that it works! -tim vince wrote:http://groups.google.com/group/web2py/browse_thread/thread/8e3e33879974663f/64f76c562366aec4?lnk=gst&q=json#64f76c562366aec4willitbe added or we'll have to wait? i think it's hard to avoid all dependency contrib / thirdparty package. as long as it's include i think that's fine. -vince-- Timothy Farrell<tfar...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)-- Timothy Farrell<tfar...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)-- Timothy Farrell<tfar...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)-- Timothy Farrell<tfar...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)

[mdipierro]

Got it. Thank you Tim. Please look again at the latest trunk. My
implementation is much slower than yours but it is
independent of the database backend and I think this is a must.
If you can you optimize it further, that is very much welcome.

Massimo

> > mdipierro wrote:where? Can you send me a patch? Massimo On Dec 29, 9:01 am, Timothy Farrell<tfarr...@swgen.com>wrote:You might want to switch back to list comprehensions. Granted the code is uglier but there are two big reasons. 1) Speed. Seehttp://www.daniweb.com/forums/thread32869.htmlfor more info. You could wrap the inner loop inner workings in a function and still reap massive benefits by not having to call list.append() n^2 times. 2) It's recommended for building lists in Py3k. The results of the above linked test are:[0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58] [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58] Profile of evenList1(), check the time consumed by the many append() calls: (ignore time consumed by the profiler itself) 50005 function calls in 0.174 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 50000 0.066 0.000 0.066 0.000 :0(append) 1 0.002 0.002 0.002 0.002 :0(range) 1 0.005 0.005 0.005 0.005 :0(setprofile) 1 0.001 0.001 0.168 0.168 <string>:1(<module>) 1 0.100 0.100 0.168 0.168 Script1.py:5(evenList1) 1 0.000 0.000 0.174 0.174 profile:0(evenList1()) 0 0.000 0.000 profile:0(profiler) Profile of evenList2(): (ignore time consumed by the profiler itself) 5 function calls in 0.018 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.002 0.002 0.002 0.002 :0(range) 1 0.000 0.000 0.000 0.000 :0(setprofile) 1 0.001 0.001 0.018 0.018 <string>:1(<module>) 1 0.015 0.015 0.017 0.017 Script1.py:15(evenList2) 1 0.000 0.000 0.018 0.018 profile:0(evenList2()) 0 0.000 0.000 profile:0(profiler) mdipierro wrote:OK. it is in trunk but slightly different. The representation is now independent on the database backend. Massimo On Dec 29, 8:04 am, Timothy Farrell<tfarr...@swgen.com>wrote:Massimo, would it help if I made a bzr branch? Timothy Farrell wrote:I've been using this in my setup (my setup has a few customizations) since the thread linked below. Just to say that it works! -tim vince wrote:http://groups.google.com/group/web2py/browse_thread/thread/8e3e33879974663f/64f76c562366aec4?lnk=gst&q=json#64f76c562366aec4willitbe added or we'll have to wait? i think it's hard to avoid all dependency contrib / thirdparty package. as long as it's include i think that's fine. -vince-- Timothy Farrell<tfarr...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)-- Timothy Farrell<tfarr...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)-- Timothy Farrell<tfarr...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)-- Timothy Farrell<tfarr...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)

[Timothy Farrell]

OK, I've got my faster method ready.  But I have a question before we lock down the API.

In your method, you have the mode parameter as either a dict or list type.  While expressing a similar thought, this is inaccurate since we're talking about a Javascript result not a Python result.  Instead the options should be 'object' or 'array'.

If you look at my method, you'll see that the mode variable is merely an indicator and not directly involved in the process (as it was in your method revision 320 line #1427).

The full diff can be seen here:
http://bazaar.launchpad.net/~tfarrell-swgen/web2py/fasterJSON/revision/323?compare_revid=320

Vince, do you have a preference with the mode API?  Anyone else?

-tim

-- 
Timothy Farrell <tfar...@swgen.com>

[mdipierro]

Thank you Tim,
your patch is in trunk!

Massimo

On Dec 29, 3:09 pm, Timothy Farrell <tfarr...@swgen.com> wrote:
> OK, I've got my faster method ready. But I have a question before we lock down the API.
> In your method, you have the mode parameter as either a dict or list type. While expressing a similar thought, this is inaccurate since we're talking about a Javascript result not a Python result. Instead the options should be 'object' or 'array'.
> If you look at my method, you'll see that the mode variable is merely an indicator and not directly involved in the process (as it was in your method revision 320 line #1427).
> The full diff can be seen here:http://bazaar.launchpad.net/~tfarrell-swgen/web2py/fasterJSON/revision/323?compare_revid=320
> Vince, do you have a preference with the mode API? Anyone else?
> -tim

> mdipierro wrote:Got it. Thank you Tim. Please look again at the latest trunk. My implementation is much slower than yours but it is independent of the database backend and I think this is a must. If you can you optimize it further, that is very much welcome. Massimo On Dec 29, 11:02 am, mdipierro<mdipie...@cs.depaul.edu>wrote:Sorry. I must have got confused. I do not think I have seen your implementation than. Please email it to me. Massimo On Dec 29, 9:22 am, Timothy Farrell<tfarr...@swgen.com>wrote:Yes, but before I do that, I want to clarify something. My first implementation results in a JSON list of objects like so (roughly):[ {field1:'field1val',field2:'field2val'}, {field1:'field1val',field2:'field2val'} ] Your reimplementation results in a JSON list of lists where the first list is the field (column) names like so: [ ['field1', 'field2'], ['field1val', 'field2val'], ['field1val', 'field2val'] ] My implementation is more useful for programming in Javascript. With your implementation (while more efficient on the wire), requires that we have to unpack the data in client-side Javascript which adds an extra step for the developer and, if the user has either IE and/or a slow computer this could add an undesirable delay for large datasets (which was the whole point of my using list comprehensions in the first place). So can I send you a patch redone with my method (JSON list of objects)? -tim mdipierro wrote:where? Can you send me a patch? Massimo On Dec 29, 9:01 am, Timothy Farrell<tfarr...@swgen.com>wrote:You might want to switch back to list comprehensions. Granted the code is uglier but there are two big reasons. 1) Speed. Seehttp://www.daniweb.com/forums/thread32869.htmlfor more info. You could wrap the inner loop inner workings in a function and still reap massive benefits by not having to call list.append() n^2 times. 2) It's recommended for building lists in Py3k. The results of the above linked test are:[0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58] [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58] Profile of evenList1(), check the time consumed by the many append() calls: (ignore time consumed by the profiler itself) 50005 function calls in 0.174 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 50000 0.066 0.000 0.066 0.000 :0(append) 1 0.002 0.002 0.002 0.002 :0(range) 1 0.005 0.005 0.005 0.005 :0(setprofile) 1 0.001 0.001 0.168 0.168 <string>:1(<module>) 1 0.100 0.100 0.168 0.168 Script1.py:5(evenList1) 1 0.000 0.000 0.174 0.174 profile:0(evenList1()) 0 0.000 0.000 profile:0(profiler) Profile of evenList2(): (ignore time consumed by the profiler itself) 5 function calls in 0.018 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.002 0.002 0.002 0.002 :0(range) 1 0.000 0.000 0.000 0.000 :0(setprofile) 1 0.001 0.001 0.018 0.018 <string>:1(<module>) 1 0.015 0.015 0.017 0.017 Script1.py:15(evenList2) 1 0.000 0.000 0.018 0.018 profile:0(evenList2()) 0 0.000 0.000 profile:0(profiler) mdipierro wrote:OK. it is in trunk but slightly different. The representation is now independent on the dat abase backend. Massimo On Dec 29, 8:04 am, Timothy Farrell<tfarr...@swgen.com>wrote:Massimo, would it help if I made a bzr branch? Timothy Farrell wrote:I've been using this in my setup (my setup has a few customizations) since the thread linked below. Just to say that it works! -tim vince wrote:http://groups.google.com/group/web2py/browse_thread/thread/8e3e33879974663f/64f76c562366aec4?lnk=gst&q=json#64f76c562366aec4willitbeadded or we'll have to wait? i think it's hard to avoid all dependency contrib / thirdparty package. as long as it's include i think that's fine. -vince-- Timothy Farrell<tfarr...@swgen.com>Computer Guy Statewide General Insura nce Agency (www.swgen.com)-- Timothy Farrell<tfarr...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)-- Timothy Farrell<tfarr...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)-- Timothy Farrell<tfarr...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)-- Timothy Farrell<tfar...@swgen.com>Computer Guy Statewide General Insurance Agency (www.swgen.com)