What gets posted to the server when a Nitrogen event is triggered?

[Ramkrishna Kulkarni]

Hello,

I've a page with over 300 buttons and thousand radio buttons. Most of these are hidden and at a time only few buttons are displayed based on user context.

This is done because most of the content does not change. Only events are triggered based on button / radio click.

Now, when an event is triggered, I see over 70KB of data being posted.

What actually gets posted to the server on a click event? Why is the post size so large?

Thanks.

[Joshua Muzaaya]

For most components, you specify a 'postback' value in its record definition. However, if you defined a postback for each of your components, i dont think that clicking on one should trigger a postback from all. Perhaps, we need more information on this use case.

--
You received this message because you are subscribed to the Google Groups "Nitrogen Project / The Nitrogen Web Framework for Erlang" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nitrogenweb...@googlegroups.com.
To post to this group, send email to nitro...@googlegroups.com.
Visit this group at https://groups.google.com/group/nitrogenweb.
For more options, visit https://groups.google.com/d/optout.

[Stuart Thackray]

I believe the page content i.e. whatever is stored is <script> Nitrogen.$set_param('pageContext', '...'
Is sent on each interaction; I may be mistaken. 

[Joshua Muzaaya]

I would guess that is executed locally, then perhaps the call ends up in an Ajax call.

--

[Ramkrishna Kulkarni]

That's correct looking at the post body. There is a huge pageContext part and tiny eventContext part.

Post body -> https://gist.github.com/rkulkarni/5afede37b1f1058807d5307e04fd8613

I thought only the eventContext would be necessary. Perhaps I'm using it wrong.

Is there an efficient way to handle my use case (i.e. loading entire page at once in the beginning and play with events)?

Ram

[Jesse Gumm]

Hey Ram,

The pageContext contains anything that might be set with wf:state(). The reason that information is relayed back to the client is so that a postback can continue at the same page state if the page has been abandoned.  This also minimizes the amount of RAM used on the server, while increasing amount of RAM used on the client, and increasing the size of the postbacks.

However, if you really need to store as much info in wf:state as you're doing, you have a few options, most involve making a custom state_handler (http://nitrogenproject.com/doc/handlers/state.html)

1) Make the state_handler store the page context in an ETS or DETS table, and rather than having to serialize the whole thing, you only have to serialize the key to the data.  You will, however, want to make sure this data gets cleaned up periodically.

2) Make the state_handler store the page context in (if you're using my refactor branch), the cache_handler, and the wf:cache functions (which have a provided TTL and then auto-clean up after themselves, as long as you set the expiration date accordingly). See https://github.com/choptastic/nitrogen_core/blob/refactor/src/wf.erl#L411-L433

There was a discussion a few months back about reworking the state handler in this way (as someone else was also wrapping a ton of data with wf:state - like something on the order of a megabyte of data), causing *massive* postbacks.

Additionally, with that many buttons, it got me thinking "I bet the postback still packages up buttons, it probably doesn't need to do that". And sure enough. I don't believe there's really a good reason to package up and send the buttons across the wire the same as textboxes, and select boxes and the like, since buttons will typically be handled using postbacks. This would cut down on the temp_xvy.temp_abc.temp_something.... spam within your postbacks.

The problem itself is illustrated by you having hundreds of buttons (where most of my own forms may have 5 or 6 buttons at most, so it isn't really a *problem* for me, but I can see how this would in general be detrimental).

I'm going to make a quick fix and push it onto both mainline and my refactor branch.  Then I'll post back here to let you know to give it a shot.

-Jesse

--

You received this message because you are subscribed to the Google Groups "Nitrogen Project / The Nitrogen Web Framework for Erlang" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nitrogenweb...@googlegroups.com.
To post to this group, send email to nitro...@googlegroups.com.
Visit this group at https://groups.google.com/group/nitrogenweb.
For more options, visit https://groups.google.com/d/optout.

--

Jesse Gumm
Owner, Sigma Star Systems
414.940.4866 || sigma-star.com || @jessegumm

[Ramkrishna Kulkarni]

Thanks a lot Jesse! Meanwhile I'll try the custom state handler approach. 

[Jesse Gumm]

Great, I'd love to see your custom state handler, since it may be worth including in mainline Nitrogen at some point as an alternative state handler for folks like yourself who might need it.

Are you indeed storing many things in the state handler as I suspected? Or perhaps using many validators (since those are also stored in the state handler)?

As for my proposed change with javascript packing up all the buttons and sending them over the wire too, here's a quick fix: https://github.com/choptastic/nitrogen_core/commit/fb603a562c7e6fa4fd569c303453ee4f535f1475

-Jesse

[Ramkrishna Kulkarni]

Thanks! I'll get back to you on what's being stored in the state by tomorrow. I'm reviewing my colleague's code.

Is there a way to decode the pageContext string? That would make things simpler to peek into.

[Jesse Gumm]

Also, I just re-read this line I wrote: "The reason that information is relayed back to the client is so that a postback can continue at the same page state if the page has been abandoned"

That phrasing is a little broken, and while I suspect you know what I mean, for anyone else reading this who might be confused what I mean by this, here's a clarification:

What it means is that the pageContext (pageState) is relayed back to the client both on initial loading and with every postback so that even if the user steps away from their computer for any length of time (leaving over a weekend, for example), as long as they've left the page up in their browser, they could get there, click a button that does a postback, and the postback will work because serialized pageContext stored on the client contains the necessary context to properly handle the postback.

-Jesse

[Jesse Gumm]

Sure.

Probably the easiest would be to just modify wf_core and output the contents of the pageContext before it gets serialized here: https://github.com/nitrogen/nitrogen_core/blob/master/src/wf_core.erl#L153

Or, you could fudge it on the Erl shell by

1) Make a fake context for the erl shell (a context is necessary for pickling and depickling)

wf_context:init_context(undefined).

2) Depickle the pageContext:

wf:depickle("zF8cGHuVucZt6FNVh0o7A0t2dBgyJKsPipb2jNiO1s0.....").

Just be sure you paste that whole pageContext into the wf:depickle function. It's checksummed, so if anything is off, it'll return undefined.

-Jesse

[Stuart Thackray]

to decode the pageContext string you do: 

wf_context:init_context(undefined).
wf:depickle(<STRING>).

It would return something like

(my...@127.0.0.1)4> wf:depickle("hWUv4yasl-2YyUTLUFQV22io3M5TKa2EJ7Tm4F7MpdbhSemkA7atSLkzYS4DbxRmDijiBy7kbQyt1JHyGLRZ5EVVN5gpcx_nsxeHUwKHTjtUmSWf7DJkiU8kLLwU0v_a6jzD2k7Fa0jtxjAFbFzt2TnG6xfBp8ylh2sAWeM6jcwjZBTgix3EKC6Jje_XQOLTClwgkW_INmMmM5a1_kdWNyVUWXo-fQy1Temtwt4F8NRzKwrEouXTz-xbMyl-6h4DVDbtkw").

[{page_context,"temp383647",index,main,[],comet},

 {handler_context,state_handler,default_state_handler,

                  undefined,[]}]

[Ramkrishna Kulkarni]

I've pasted the decoded page context here https://gist.github.com/rkulkarni/9d6779d8d9be53351158d8969c2be89d

...

{button5791a417f9581f88b0db702daf1df8f7,50},

{button5791a417f9581f88b0db702daf184e00,49},

{button5791a417f9581f88b0db702daf19717d,48},

{button5791a417f9581f88b0db702daf19b8f7,47},

...

As you can see, all the button ids are being stored which is pushing up the size. Is this really needed?

[Jesse Gumm]

It's probably not necessary, but without seeing the code, it's hard to determine *why* the button IDs are being stored.  There's nothing inherent in the button implementation that stores the ids manually.

But generally, I haven't come across a whole lot of reason to store element IDs in the page state. Rather, if a postback needs to refer to a specific ButtonID, I include it in the button's postback.  Like to have a button delete itself through a postback when using a randomly generated ID, I might do something like this:

ID = wf:temp_id(),

#button{id=ID, postback={delete_button, ID}}.

...

event({delete_button, ID}) ->

    wf:remove(ID).

ALLLSO, since it looks like you're randomly generating IDs for your buttons, I'd look into wf:temp_id(), particularly if you're using Erlang 18+ and my 'refactor' branch. If you're using those two, then wf:temp_id() uses erlang:unique_integer() to generate IDs (which is more-or-less guaranteed to be unique).

-Jesse

[Ramkrishna Kulkarni]

Ah! Sorry, my bad. The button ids are explicitly being stored in the state :(

I might not need a custom state handler at all now but I'll try on using Erlang maps. Feels like a right fit here.

Thanks a ton!