Ignore query parameters?

[Bill Kocik]

Is there a way to get fakeweb to ignore query parameters such that if
I do FakeWeb.register(:get, 'http://example.com', :string =>
'Huzzah!') then each of http://example.com,
http://example.com?foo=bar, http://example.com?page=42, etc. would all
return the string "Huzzah!" without having to be individually
registered?

I just typed all that and then found this:
http://github.com/freelancing-god/fakeweb/commit/50b9a17ae5d020cdbaf5237e848f91561a839858

...suggesting that it's not in the core version. Any chance of doing a
pull of that feature? Pretty please?

--
Bill Kocik

http://bkocik.net

[Chris Kampmeier]

Hey Bill,

I was going to pull that patch, but then I started working on regex
support for registering URIs instead, which seems like a more general-
purpose solution. It also seems more elegant, since each URI could
support ignoring query parameters or not (depending on how you write
the regex), and you could make one query parameter significant but
ignore the rest, etc. You could also ignore certain parts of a path,
the port number, a file extension, basic-auth userinfo strings, and so
on.

The problem is that this is actually a little complex, since a lot of
FakeWeb depends on string-manipulation of the registered URIs right
now. So I'm adding a ton of tests to the suite (and doing some
implementation refactoring along the way) to make sure I don't break
anything.

I'm interested, though, can you tell me about your use case for
ignoring query params? Most of the APIs I've stubbed with FakeWeb
either don't use many query params, or they're pretty significant when
they do. But I know there are some weird query-param-based APIs out
there (ahem FogBugz). So I'd like to hear more about what you're
doing!

Thanks-
Chris

On May 8, 9:49 pm, Bill Kocik <bko...@gmail.com> wrote:
> Is there a way to get fakeweb to ignore query parameters such that if
> I do FakeWeb.register(:get, 'http://example.com', :string =>

> 'Huzzah!') then each ofhttp://example.com,http://example.com?foo=bar,http://example.com?page=42, etc. would all

> return the string "Huzzah!" without having to be individually
> registered?
>

> I just typed all that and then found this:http://github.com/freelancing-god/fakeweb/commit/50b9a17ae5d020cdbaf5...

[Mike Doel]

On May 11, 2009, at 3:59 AM, Chris Kampmeier wrote:

> I was going to pull that patch, but then I started working on regex
> support for registering URIs instead, which seems like a more general-
> purpose solution

Huzzah! That will be awesome.

Mike

[Bill Kocik]

On Mon, May 11, 2009 at 3:59 AM, Chris Kampmeier
<ChrisGK...@gmail.com> wrote:

> I'm interested, though, can you tell me about your use case for
> ignoring query params? Most of the APIs I've stubbed with FakeWeb
> either don't use many query params, or they're pretty significant when
> they do. But I know there are some weird query-param-based APIs out
> there (ahem FogBugz). So I'd like to hear more about what you're
> doing!

I'm hitting the Twitter API using John Nunemaker's twitter gem, and
depending on what I'm doing it might pass a page param, or a "since"
param, or any of a number of others, but for the most part they're not
relevant to my tests, so I'd rather not have to fake all the
possibilities, I'd rather just be able to say "this is the end point -
if there are query parameters attached, ignore them".

I think regular expressions would be an excellent solution for this.

[Matt Patterson]

On May 11, 8:59 am, Chris Kampmeier <ChrisGKampme...@gmail.com> wrote:
>
> I was going to pull that patch, but then I started working on regex
> support for registering URIs instead, which seems like a more general-
> purpose solution. It also seems more elegant, since each URI could
> support ignoring query parameters or not (depending on how you write
> the regex), and you could make one query parameter significant but
> ignore the rest, etc. You could also ignore certain parts of a path,
> the port number, a file extension, basic-auth userinfo strings, and so
> on.

Aha! I started using FakeWeb a couple of days ago to try and make a
proper go of cucumber scenarios for an app I'm working on which hits
Vimeo's API. (As an aside, I have a small set of Cucumber scenarios
which actually hit the API for real and a larger set which fake out
HTTP for better control).

My problem faking it out has been that there are a number of query
string params which are significant, and a number which aren't, and
which change from request to request (things like MD5 sums of all the
other params). I want to only specify the significant ones because the
insignificant ones change from request to request. So far, so normal.

The problem I foresee with a regex based solution is that because
clients (HTTParty for example) often construct the query string from a
parameter hash, there's not a defined order for the query string's
params, and the generated URI doesn't have a predictable order. A
regex for that is going to be horrid.

> The problem is that this is actually a little complex, since a lot of
> FakeWeb depends on string-manipulation of the registered URIs right
> now. So I'm adding a ton of tests to the suite (and doing some
> implementation refactoring along the way) to make sure I don't break
> anything.

I was wondering about pulling FakeWeb::Responders out of a URI-keyed
hash, so you could iterate over the responders and have the matching
intelligence there instead, then you could have a DSLish block
matching syntax for the complex stuff

FakeWeb.register_uri('http://blah.net/gubbins', :response =>
response_object)

FakeWeb.register_uri(/http:\/\/blah.net/abc[0-9]/, :response =>
response_object)

FakeWeb.register_uri do
host 'blah.net'
path '/gubbins'
matching_query_params 'param' => 'value', 'other_param' =>
'other_value'
# or maybe its inverse
ignoring_query_params 'pointless' => 'timestamp'

# or maybe something like this
uri 'http://blah.net/gubbins/'
ignoring_unmarked_query_params
marked_query_params 'param' => 'value', 'other_param' =>
'other_value'

# which would then give you this
uri 'http://blah.net/gubbins/'
marked_query_params_excluded # like grep -v
marked_query_params 'param' => 'value', 'other_param' =>
'other_value'

response response_object
end

> I'm interested, though, can you tell me about your use case for
> ignoring query params?

For Vimeo, the API endpoint is a query param rather than a URI: all
the API methods have the same host, protocol and path portions. there
are also checksum-like params, auth tokens (which vary between test
account to the production account), and various params related to
pagination and response detail.

Here's a full (anonymised but with the same number of characters) URI:

http://vimeo.com/api/rest?auth_token=1a2b3c4d5e6f708a9b0c1d2e3f405a6b&fullResponse=&page=&api_sig=2b3c4d5e6f708a9b0c1d2e3f405a6b7c&method=vimeo.videos.getUploadedList&api_key=4d5e6f708a9b0c1d2e3f405a6b7c8d9e&user_id=anonymised&per_page=

A bit of a pig to wrangle...

Anyway. I'm more than happy to contribute code and test coverage.

Cheers,

Matt

[ajsharp]

My issue has been with the Google geocoding service I'm using in a
model. I'm generating instances of this model frequently in my tests
using Factory Girl. The Google geocoding API relies heavily on
parameters, and it would be nice not to have to register each
different type of factory with FakeWeb. I'm not sure on the best way
to implement this (I've only glanced over the FakeWeb Registry class
code), but I just thought I would share my use case, which I assume is
a fairly common one. Other than that, thanks for the great gem!

> http://vimeo.com/api/rest?auth_token=1a2b3c4d5e6f708a9b0c1d2e3f405a6b...

[Mike Doel]

On May 16, 2009, at 8:00 PM, ajsharp wrote:

> My issue has been with the Google geocoding service I'm using in a
> model. I'm generating instances of this model frequently in my tests
> using Factory Girl. The Google geocoding API relies heavily on
> parameters, and it would be nice not to have to register each
> different type of factory with FakeWeb. I'm not sure on the best way
> to implement this (I've only glanced over the FakeWeb Registry class
> code), but I just thought I would share my use case, which I assume is
> a fairly common one. Other than that, thanks for the great gem!

If you're using geokit for this, you don't need to use FakeWeb.
Here's what I do:

1) I have a geocode_mocker.rb file whose contents you can see at:

http://gist.github.com/112850

This provides methods for mocking out the google geocoding call with
the contents of an xml fixture file on disk.

2) I have a rake task that takes an address and generates one of those
fixture files. You can see it at:

http://gist.github.com/112853

3) In my spec_helper.rb file, I just mock out addresses that I use in
various places in my test suite like:

stub_geocode_lookup("Miami, Fl","miamifl.xml")

I imagine there are probably more elegant ways to solve this, but it
works well enough for me.

Mike

[ajsharp]

Very cool, thanks for posting this.