Simplifying and fixing parameter parsing in Rails (Or: complex forms don't have to suck forever)

[Thomas Lee]

Hi,

A few months ago I fixed a Rails bug that was yielding weird, unexpected
results when parsing form parameter names: effectively HWIA semantics
were screwing up UrlEncodedPairParser's parameter parsing when dealing
with nested values. This fixed the defect in question -- and thus solved
my immediate problem -- it raised wider questions about the actual
correctness and behavior of Rails' current parameter parsing code.

A colleague of mine recently ran into a related problem when dealing
with collections of records in Rails forms, which prompted me to finally
do something about the parameter parsing behavior. Now, while I'm sure
we're not the only ones hitting this wall, I'm aware that proposals to
change to the parameter parsing semantics in Rails is likely to be met
with a little caution, hesitation -- possibly even terror. :) With that
in mind, I've put up a plugin so everybody can give this a go without
having to apply any nasty patches:

http://www.vector-seven.com/git/rails/plugins/form_collections.git

Please see the README (attached) for the full details.

Be gentle, this is an early release that has been cobbled together over
the course of a few nights. We may not have everything right here, but
that's why it's being proposed as a plugin rather than yet another issue
in the tracker. It's hoped this will yield a wider discussion around the
semantics of parameter parsing in core Rails, because -- as it stands --
things can get a little screwy.

Looking forward to your feedback, suggestions and/or patches. If this
breaks your code in ways you don't expect, or doesn't quite work how you
would like/need it to, or if it's missing a feature that you'd kill to
have, I'd love to know.

Please be vocal. :)

Cheers,
Tom

[Michael Koziarski]

Hi thomas,

Sorry to top reply so briefly, but have you seen what david dollar and
co have been discussing in the recent :accessible thread?

http://groups.google.com/group/rubyonrails-core/t/4049b4b313fa8be2

Sounds like your plugin here has some changes which could get merged
along with his stuff.

> FormCollections
> ===============
>
> This plugin effectively rewrites UrlEncodedPairParser and provides the
> following features/changes:
>
> 1. The parameter parsing algorithm is much simpler
>
> There's all sorts of wacky stuff happening in the current implementation of
> UrlEncodedPairParser#parse. This plugin does away with that. A stack is no
> longer used during the parse. Parsing an Array vs. parsing a Hash is no longer
> all that different. See lib/patch_url_encoded_pair_parser.rb.
>
> 2. Parsing "a[b][0][c]=6" yields {"a" => {"b" => [{"c" => "6"}]}}
>
> Previously this would be parsed to: {"a" => {"b" => {"0" => {"c" => "6"}}}}
>
> In other words, parameters that were formerly treated as hashes with a numeric
> index are now actually arrays.
>
> This is important, because the order in which form fields are present may be
> important to the back-end processing code. Here, the array in "b" will
> preserve the ordering specified -- generating an Array rather than a Hash for
> numeric indices. Most other scenarios should essentially work as before, with
> the exception of a few error cases (e.g. parsing "a/b@[c][d[e][]=f" yields
> {"a/b@" => {"c" => {"d[e" => ["f"]}}} instead of {"a/b@" => {"c" => {}}}).
>
> The main problem here is that if an array is instantiated with a value at
> index 1000000, then we'll have 999999 nil elements. I figure this can easily
> be worked around with an Array-like structure that consumes only as much memory
> as it needs but otherwise acts as an Array, or a hard limit set via
> configuration variable.
>
> 3. fields_for now treats Arrays properly ...
>
> Example:
>
> <% fields_for @post.comments do |comment_form, comment| %>
> <%= comment.new_record? ? "" : comment_form.hidden_field(:id) %>
> <%= comment_form.hidden_field :post_id %>
> <%= comment_form.text_field :content %>
> <hr />
> <% end %>
>
> This will generate the following HTML for an array with two elements (one a new_record?, the other existing):
>
> <input id="comments_0_id" name="comments[0][id]" type="hidden" value="1" />
> <input id="comments_0_post_id" name="comments[0][post_id]" type="hidden" value="1" />
> <input id="comments_0_content" name="comments[0][content]" size="30" type="text" value="a test comment" />
> <hr />
> <input id="comments_1_id" name="comments[1][post_id]" type="hidden" value="1" />
> <input id="comments_1_content" name="comments[1][content]" size="30" type="text" value="" />
> <hr />
>
>
> Example
> =======
>
> Just install the plugin, and the new behavior should already be in effect.
>
> Copyright (c) 2008 Thomas Lee <t...@vector-seven.com>, released under the MIT license
>
>

--
Cheers

Koz

[Thomas Lee]

Michael Koziarski wrote:
> Hi thomas,
>
> Sorry to top reply so briefly, but have you seen what david dollar and
> co have been discussing in the recent :accessible thread?
>
> http://groups.google.com/group/rubyonrails-core/t/4049b4b313fa8be2
>
> Sounds like your plugin here has some changes which could get merged
> along with his stuff.
>
>

Sure does. Thanks for pointing that out. Looks like two sides of the
same coin. :)

Cheers,
T