mapToParameters

[Kevin Dangoor]

I sent a little patch to Bob earlier today, forgetting that there's
now a mailing list. As I was actively using the code in question, I
came across something else I wanted it to do... so, I'm sending the
revised patch to the list now...

Index: tests/test_MochiKit-Base.html
===================================================================
--- tests/test_MochiKit-Base.html (revision 226)
+++ tests/test_MochiKit-Base.html (working copy)
@@ -12,7 +12,7 @@
try {

// Counting the number of tests is really lame
- plan({'tests': 99});
+ plan({'tests': 104});

// test bind
var not_self = {"toString": function () { return "not self"; } };
@@ -325,6 +325,17 @@
is( repr(o), "NAME", "NAME protocol (func)" );

is( repr(MochiKit.Base.nameFunctions),
"MochiKit.Base.nameFunctions", "test nameFunctions" );
+
+ // test URL parameters handling
+
+ is( URLencode("1+2=2"), "1%2B2%3D2", "URL encoding")
+ is( mapToParameters({"a" : 1}), "a=1", "mapToParameters with 1 item" );
+ is( mapToParameters({"a" : "hello", "b": "goodbye"}), "a=hello&b=goodbye",
+ "mapToParameters with multiple items" );
+ is( mapToParameters({"spacey" : "has some spaces"}),
"spacey=has%20some%20spaces",
+ "mapToParameters with spaces" );
+ is( mapToParameters({"nullparam" : null}), "", "null parameters
are removed" );
+
// Done!

ok( true, "test suite finished!");
Index: doc/rst/MochiKit/Base.rst
===================================================================
--- doc/rst/MochiKit/Base.rst (revision 226)
+++ doc/rst/MochiKit/Base.rst (working copy)
@@ -779,7 +779,25 @@
nameFunctions(namespace);
assert( namespace.Dude.NAME == 'Awesome.Dude' );

+``URLencode(unencoded)``:

+ Converts a string into a URL-encoded string. Note that, in this
+ implementation, spaces are converted to %20 instead of "+". e.g.::
+
+ assert( URLencode("1+2=2") == "1%2B2%3D2")
+
+``mapToParameters(parameterMap)``:
+
+ Converts a JavaScript hash mapping into a URL string. Anything in the map
+ with a null value is not included in the final set of parameters. If you
+ are appending this to a URL, you will need to add the "?" at the
+ beginning. If you are appending this to a URL that already contains
+ parameters, you will need to add the "&" at the beginning. e.g.::
+
+ assert( mapToParameters({"a" : "hello", "b": "goodbye"}) ==
+ "a=hello&b=goodbye" );
+
+
Authors
=======

Index: MochiKit/Base.js
===================================================================
--- MochiKit/Base.js (revision 226)
+++ MochiKit/Base.js (working copy)
@@ -976,6 +976,32 @@
}
}

+MochiKit.Base.URLencode = function(unencoded) {
+ /***
+ Returns a string that is usable for GET parameters in a URL.
+ ***/
+ return escape(unencoded).replace(/\+/g, '%2B')
+ .replace(/\"/g,'%22').replace(/\'/g, '%27');
+}
+
+MochiKit.Base.mapToParameters = function(pmap) {
+ /***
+ Converts a map of simple name, value pairs into a
+ string that is properly encoded for use in a URL.
+ ***/
+ urlstring = "";
+ for (key in pmap) {
+ if (pmap[key] == null) {
+ continue;
+ }
+ if (urlstring) {
+ urlstring += "&";
+ }
+ urlstring += key + "=" + MochiKit.Base.URLencode(pmap[key]);
+ }
+ return urlstring;
+}
+
MochiKit.Base.EXPORT = [
"clone",
"extend",
@@ -1020,7 +1046,9 @@
"objMax",
"objMin",
"nodeWalk",
- "zip"
+ "zip",
+ "mapToParameters",
+ "URLencode"
];

MochiKit.Base.EXPORT_OK = [
Index: MochiKit/Async.js
===================================================================
--- MochiKit/Async.js (revision 226)
+++ MochiKit/Async.js (working copy)
@@ -562,6 +562,17 @@
return MochiKit.Async.sendXMLHttpRequest(req);
};

+MochiKit.Async.doAjaxRequest = function(optionsInput) {
+ options = {
+ method : "GET",
+ async : true,
+ decode : null,
+ parameters : null
+ };
+ MochiKit.Base.update(options, optionsInput);
+
+}
+
MochiKit.Async.loadJSONDoc = function (url) {
/***

[Ian Bicking]

Kevin Dangoor wrote:
> +MochiKit.Base.mapToParameters = function(pmap) {
> + /***
> + Converts a map of simple name, value pairs into a
> + string that is properly encoded for use in a URL.
> + ***/
> + urlstring = "";
> + for (key in pmap) {
> + if (pmap[key] == null) {
> + continue;
> + }
> + if (urlstring) {
> + urlstring += "&";
> + }
> + urlstring += key + "=" + MochiKit.Base.URLencode(pmap[key]);
> + }
> + return urlstring;
> +}
> +

Incidentally it's legal for keys to be non-URL safe (like in the PHP
"array_var[]" convention), so they should be encoded too.

--
Ian Bicking / ia...@colorstudy.com / http://blog.ianbicking.org

[Kevin Dangoor]

On 8/2/05, Ian Bicking <ia...@colorstudy.com> wrote:
> Incidentally it's legal for keys to be non-URL safe (like in the PHP
> "array_var[]" convention), so they should be encoded too.

I had debated about that. I should've guessed that there would be
folks that would use non-URL safe variables (and I should've guessed
WHO they'd be!)

Adding a URLencode call around the key would be a good idea.

Kevin

[Bob Ippolito]

On Aug 2, 2005, at 8:56 AM, Kevin Dangoor wrote:

> +``URLencode(unencoded)``:
>
> + Converts a string into a URL-encoded string. Note that, in this
> + implementation, spaces are converted to %20 instead of "+".
> e.g.::
> +
> + assert( URLencode("1+2=2") == "1%2B2%3D2")

This one looks good, but I think I'd like to call it urlEncode instead.

> +``mapToParameters(parameterMap)``:
> +
> + Converts a JavaScript hash mapping into a URL string. Anything
> in the map
> + with a null value is not included in the final set of
> parameters. If you
> + are appending this to a URL, you will need to add the "?" at the
> + beginning. If you are appending this to a URL that already
> contains
> + parameters, you will need to add the "&" at the beginning. e.g.::

I don't really like the name of this one.. In JavaScript, every
object is a string map and there's no particularly special object to
use as a map. I think we should shoot for something close to the RFC
1630 <http://www.w3.org/Addressing/rfc1630.txt> name for what we're
assembling here: a query string (yes, I know that query strings don't
necessarily imply field value pairs.. but if you just wanted a string
you'd just use a string).

What about just calling it queryString(parameters)?

Also, for those poor souls using another library that modifies
Object.prototype, perhaps there should either be an alternate form
that takes (keys, values)? Or maybe we should document that function
values are skipped? Or I guess we could skip keys such that
parameters[key] === Object.prototype[key].

-bob

[Kevin Dangoor]

On 8/2/05, Bob Ippolito <b...@redivi.com> wrote:
> This one looks good, but I think I'd like to call it urlEncode instead.

Sounds good.

> I don't really like the name of this one.. In JavaScript, every
> object is a string map and there's no particularly special object to
> use as a map. I think we should shoot for something close to the RFC
> 1630 <http://www.w3.org/Addressing/rfc1630.txt> name for what we're
> assembling here: a query string (yes, I know that query strings don't
> necessarily imply field value pairs.. but if you just wanted a string
> you'd just use a string).
>
> What about just calling it queryString(parameters)?

That sounds fine to me. WIth all of the Java I've done, calling it
mapToParameters seemed logical, but I also knew that felt a bit funny
as far as JavaScript is concerned.

> Also, for those poor souls using another library that modifies
> Object.prototype, perhaps there should either be an alternate form
> that takes (keys, values)? Or maybe we should document that function
> values are skipped? Or I guess we could skip keys such that
> parameters[key] === Object.prototype[key].

Now who would use a library like that? :)

I like your last suggestion. It picks up the specific behavior we're
looking to avoid.

Kevin

p.s. I had a chance today to replace a piece of code that used string
concatenation with the DOM module... the DOM module is sooo much
nicer.

[Ian Bicking]

Bob Ippolito wrote:
> Also, for those poor souls using another library that modifies
> Object.prototype, perhaps there should either be an alternate form that
> takes (keys, values)? Or maybe we should document that function values
> are skipped? Or I guess we could skip keys such that parameters[key]
> === Object.prototype[key].

What about skipping when (typeof(value) == 'function')?

[Bob Ippolito]

On Aug 2, 2005, at 10:58 AM, Ian Bicking wrote:

>
> Bob Ippolito wrote:
>
>> Also, for those poor souls using another library that modifies
>> Object.prototype, perhaps there should either be an alternate
>> form that takes (keys, values)? Or maybe we should document that
>> function values are skipped? Or I guess we could skip keys such
>> that parameters[key] === Object.prototype[key].
>>
>
> What about skipping when (typeof(value) == 'function')?

That's what I meant by "function values are skipped". After thinking
about it a little more, it would even have the added "benefit" of
pandering to idiots who like to use Array instances as parameter
objects::

// because this instantiation is soooo much clearer than the
equivalent syntax
var x = new Array();
// .. especially when you really wanted an Object instance instead!
x["foo"] = "bar";

I think these are the same people who actually like ActionScript
2.0. All the dumb things about JavaScript PLUS a broken type system
where you are encouraged to manually throw type declarations
everywhere despite the fact that there is no benefit to doing so at
all (barring the "compiler finding your bugs").

So I guess function skipping it is, since it's extremely unlikely
that non-functions will get hacked into built-in prototypes, even
when using the dumbest of JavaScript libraries.

-bob

[Bob Ippolito]

On Aug 2, 2005, at 8:56 AM, Kevin Dangoor wrote:

>

> I sent a little patch to Bob earlier today, forgetting that there's
> now a mailing list. As I was actively using the code in question, I
> came across something else I wanted it to do... so, I'm sending the
> revised patch to the list now...

I ended up with this:

- Added three new functions to MochiKit.Base for dealing with URL query
strings: urlEncode, queryString, parseQueryString

Let me know if you think these APIs are OK for what you're doing.


``urlEncode(unencoded)``:

Converts a string into a URL-encoded string. Note that, in this

implementation, spaces are converted to %20 instead of "+". e.g.::

assert( URLencode("1+2=2") == "1%2B2%3D2");


``queryString(names, values)``:

Creates a URL query string from a pair of array-like objects
representing
``names`` and ``values``. Each name=value pair will be URL
encoded by
``urlEncode``. e.g.::

var keys = ["foo", "bar"];
var values = ["value one", "two"];
assert( queryString(keys, values) == "foo=value%
20one&bar=two" );

Alternate form:
``queryString({name: value, ...})``

Note that when using the alternate form, the order of the
name=value
pairs in the resultant query string is dependent on how the
particular
JavaScript implementation handles ``for (..in..)`` property
enumeration.

When using the alternate form, name=value pairs with
``typeof(value) == "function"`` are ignored. This is a
workaround for the
case where a poorly designed library has modified
``Object.prototype``
and inserted "convenience functions".


``parseQueryString(encodedString[, useArrays=false])``:

Parse a name=value pair URL query string into an object with a
property
for each pair. e.g.::

var args = parseQueryString("foo=value%20one&bar=two");
assert( args.foo == "value one" && args.bar == "two" );

If you expect that the query string will reuse the
same name, then give ``true`` as a second argument, which will
use arrays to store the values. e.g.::

var args = parseQueryString("foo=one&foo=two");
assert( args.foo[0] == "one" && args.foo[1] == "two" );

[Kevin Dangoor]

On 8/2/05, Bob Ippolito <b...@redivi.com> wrote:

> ``parseQueryString(encodedString[, useArrays=false])``:
>
> Parse a name=value pair URL query string into an object with a
> property
> for each pair. e.g.::
>
> var args = parseQueryString("foo=value%20one&bar=two");
> assert( args.foo == "value one" && args.bar == "two" );

They all look good to me. I debated about writing the parse function,
but I didn't have an immediate need, so I didn't pursue it. Thanks for
adding it! That makes a nice, complete set.

(And, when I do have a need for it, I'll be all set!)

Kevin

[Kevin Dangoor]

I just switched my code to use this and discovered that one piece of
behavior that I had put in was lost in translation. My original
version of this function would drop parameters for which the value was
null, which made certain things handy...

I have a function called "reloadWithParams" that update()s a hash of
parameters and then calls document.location=queryString(). If the user
clicks a link that needs to make a parameter go away, I can just pass
in {"thatparam" : null}.

Any objections to that behavior? I'm figuring that if you really want
to send "null" to the server, you can just say {"thatparam" : "null"}

Kevin

On 8/2/05, Bob Ippolito <b...@redivi.com> wrote:

[Bob Ippolito]

On Aug 9, 2005, at 1:52 AM, Kevin Dangoor wrote:

>
> I just switched my code to use this and discovered that one piece of
> behavior that I had put in was lost in translation. My original
> version of this function would drop parameters for which the value was
> null, which made certain things handy...
>
> I have a function called "reloadWithParams" that update()s a hash of
> parameters and then calls document.location=queryString(). If the user
> clicks a link that needs to make a parameter go away, I can just pass
> in {"thatparam" : null}.
>
> Any objections to that behavior? I'm figuring that if you really want
> to send "null" to the server, you can just say {"thatparam" : "null"}

Ok, that should be adopted now, try it out and see if it does what
you want.

-bob

[Matt Goodall]

The signature of queryString seems odd to me. Why not accept a list of
(name,value) pairs instead of two (hopefully) parallel arrays?

I suspect this is going to cause a few gasps ;-) but if order is
unpredictable for queryString's alternate form why support it? At some
point, that's just going to cause surprise and result in x-browser
problems?

Also, it looks like (i.e. I have not tried it) queryString discards
anything where the value is missing. Query parameters with no value are
legitimate and really should be allowed. Hmm, the same is true when
parsing query strings so it looks like parseQueryString may be
incorrect too.

Cheers, Matt

[Matt Goodall]

I don't actually use MochiKit right now so I don't feel qualified to
comment like this. However ...

Personally, I would say parseQueryString's useArrays should default to
true - that's how URLs work - and to explicitly turn it off.

I would also prefer to get a list of (name,value) pairs back, i.e. in
the correct order. It shouldn't be difficult to go from that to a
dict-like object if that's a more convenient representation for the
task.

Cheers, Matt

[Kevin Dangoor]

Hi,

On 8/11/05, Matt Goodall <matt.g...@gmail.com> wrote:
> The signature of queryString seems odd to me. Why not accept a list of
> (name,value) pairs instead of two (hopefully) parallel arrays?

Seems like a reasonable suggestion. I've been using the hash form, so
I don't have a feel for one being better than the other.

> I suspect this is going to cause a few gasps ;-) but if order is
> unpredictable for queryString's alternate form why support it? At some
> point, that's just going to cause surprise and result in x-browser
> problems?

Because the order of query parameters is irrelevant on the server side
in most cases. Hashes are easy to update, which makes dynamic
modification of query parameters quite easy.

> Also, it looks like (i.e. I have not tried it) queryString discards
> anything where the value is missing. Query parameters with no value are
> legitimate and really should be allowed. Hmm, the same is true when
> parsing query strings so it looks like parseQueryString may be
> incorrect too.

You *should* be able to give a value of '' to have an empty value
appear in the query string. In the case of null values, there is a
choice between three possible actions:

1) drop the parameter
2) use empty string
3) use the word "null"

It was doing #3, but I requested #1 because it made dynamic updates easier.

http://groups-beta.google.com/group/mochikit/msg/7491854844aac6fa?hl=en&

Kevin

[Bob Ippolito]

On Aug 11, 2005, at 3:33 AM, Matt Goodall wrote:

> The signature of queryString seems odd to me. Why not accept a list of
> (name,value) pairs instead of two (hopefully) parallel arrays?

I expect most people will use the alternate form. I should probably
change the documentation and swap the two. It was just slightly
cleaner to do it this way, and it's probably more likely that your
data starts out as two arrays anyway.

> I suspect this is going to cause a few gasps ;-) but if order is
> unpredictable for queryString's alternate form why support it? At some
> point, that's just going to cause surprise and result in x-browser
> problems?

It's very common and convenient to implement query string creation
this way. It's highly unlikely that an endpoint is going to care.
If it is going to care, then use the names, values form.

> Also, it looks like (i.e. I have not tried it) queryString discards
> anything where the value is missing. Query parameters with no value
> are
> legitimate and really should be allowed. Hmm, the same is true when
> parsing query strings so it looks like parseQueryString may be
> incorrect too.

The HTML spec says that browsers may omit any field with no value,
unless it's input type="hidden" <http://www.w3.org/MarkUp/html-spec/
html-spec_8.html#SEC8.2.1>. If you want "foo=" in your query string,
give it a value of "" (anything but null, undefined, or a function).
It's not doing anything *that* surprising here.

Query parameters that aren't compromised of key/value pairs are not
strictly valid application/x-www-form-urlencoded. If you have some
funky usage of query strings, parse them yourself :) I could be
convinced to implement some non-strict mode if there were use cases
for it.

-bob

[Matt Goodall]

Kevin Dangoor wrote:
> Hi,
>
> On 8/11/05, Matt Goodall <matt.g...@gmail.com> wrote:
> > The signature of queryString seems odd to me. Why not accept a list of
> > (name,value) pairs instead of two (hopefully) parallel arrays?
>
> Seems like a reasonable suggestion. I've been using the hash form, so
> I don't have a feel for one being better than the other.
>
> > I suspect this is going to cause a few gasps ;-) but if order is
> > unpredictable for queryString's alternate form why support it? At some
> > point, that's just going to cause surprise and result in x-browser
> > problems?
>
> Because the order of query parameters is irrelevant on the server side
> in most cases. Hashes are easy to update, which makes dynamic
> modification of query parameters quite easy.

Agreed but I don't like "in most cases" much. But anyway, this is an
alternative form so I, too, can use it when it makes things easier and
I know the server won't care.

Perhaps there is really a need for a URL class of some sort so that
things can be both correct and easy? Nevow has this (although it's not
perfect) and it's a great thing.

>
> > Also, it looks like (i.e. I have not tried it) queryString discards
> > anything where the value is missing. Query parameters with no value are
> > legitimate and really should be allowed. Hmm, the same is true when
> > parsing query strings so it looks like parseQueryString may be
> > incorrect too.
>
> You *should* be able to give a value of '' to have an empty value
> appear in the query string. In the case of null values, there is a
> choice between three possible actions:
>
> 1) drop the parameter
> 2) use empty string
> 3) use the word "null"

4) just add the name to the query string with no '=' or value

i.e. (using the alternate queryString form ;-)) ...

queryString({foo:null, bar: 'wibble'})

gives:

foo&bar=wibble

[Bob Ippolito]

On Aug 11, 2005, at 3:45 AM, Matt Goodall wrote:

> I don't actually use MochiKit right now so I don't feel qualified to
> comment like this. However ...

Well if it makes you feel any better, I'm not using queryString/
parseQueryString in any production code yet either :) I do know that
at least two people are, though.

> Personally, I would say parseQueryString's useArrays should default to
> true - that's how URLs work - and to explicitly turn it off.

I'm going to call practicality beats purity here. You rarely ever
want anything but [0]. If you do expect to see repeated arguments,
you can ask for that since you're calling it directly. If you were
passing around a parsed query string that you don't know anything
about (e.g. in the implementation of Twisted's HTTP Request), then
you should give it to the user as arrays. However, since the user is
explicitly asking for the parse, the default behavior is the common
case.

> I would also prefer to get a list of (name,value) pairs back, i.e. in
> the correct order. It shouldn't be difficult to go from that to a
> dict-like object if that's a more convenient representation for the
> task.

I'm not sold. If you want to write extra code to specifically handle
fragile query strings, do it yourself. Prior art says you don't
actually want what you're asking for (cgi.parse_qs,
twisted.web.parse_qs, etc.).

-bob

[Bob Ippolito]

On Aug 11, 2005, at 4:08 AM, Matt Goodall wrote:

>
> Kevin Dangoor wrote:
>
>> Hi,
>>
>> On 8/11/05, Matt Goodall <matt.g...@gmail.com> wrote:
>>
>>> The signature of queryString seems odd to me. Why not accept a
>>> list of
>>> (name,value) pairs instead of two (hopefully) parallel arrays?
>>>
>>
>> Seems like a reasonable suggestion. I've been using the hash form, so
>> I don't have a feel for one being better than the other.
>>
>>
>>> I suspect this is going to cause a few gasps ;-) but if order is
>>> unpredictable for queryString's alternate form why support it? At
>>> some
>>> point, that's just going to cause surprise and result in x-browser
>>> problems?
>>>
>>
>> Because the order of query parameters is irrelevant on the server
>> side
>> in most cases. Hashes are easy to update, which makes dynamic
>> modification of query parameters quite easy.
>>
>
> Agreed but I don't like "in most cases" much. But anyway, this is an
> alternative form so I, too, can use it when it makes things easier and
> I know the server won't care.

I like functions to do what you want in most cases by default. If
you want the flexibility, you're going to be writing more code
anyway, so adding another ", true" isn't going to sting that much.

> Perhaps there is really a need for a URL class of some sort so that
> things can be both correct and easy? Nevow has this (although it's not
> perfect) and it's a great thing.

Right now that's YAGNI. In my experience, you simply don't do all
that much crazy stuff with URLs in typical client-side JavaScript
applications. If you're writing an application where a URL class
would make life that much better, write one and contribute it.

If you're writing some kind of server with JavaScript, you should
probably buy a Python book and see a psychiatrist (not necessarily in
that order) ;)

-bob