Returning HTTPResponse vs returning bottle.response
Joe Zhang
return HTTPResponse(status=200, body=json.dumps({'key':'value'})
However, I am also using an after_request hook, which tries to set some headers on the response by referring to bottle.response , and I found that inside the after_request hook code, it can no longer access that same HTTPResponse object, because whatever is referenced by bottle.response is a brand new response. The object as returned by HTTPResponse is lost. The is made complicated by the fact that the server for my bottle instance is a multi-threaded http server with a threadpool.
Is it because bottle.response references LocalResponse, and is therefore 'thread-local' ? What can I do so that my after_request hook can still access the same HTTPResponse object?
I can think of 2 ways:
1) Instead of returning HTTPResponse, I just return a dictionary as the body: return json.dumps({'key','value'}). But how can I return a status code other than 200 now?
2) bottle.response.status = 200; bottle.response.body = json.dumps({'key':'value'}); return bottle.response instead.
Is 2) thread-safe? Will it cause issues for multiple requests hitting the server and the returned responses get mixed up in the after_request hook?
Marcel Hellkamp
One common way to returning from a route is
return HTTPResponse(status=200, body=json.dumps({'key':'value'})
However, I am also using an after_request hook, which tries to set some headers on the response by referring to bottle.response , and I found that inside the after_request hook code, it can no longer access that same HTTPResponse object, because whatever is referenced by bottle.response is a brand new response. The object as returned by HTTPResponse is lost. The is made complicated by the fact that the server for my bottle instance is a multi-threaded http server with a threadpool.
Is it because bottle.response references LocalResponse, and is therefore 'thread-local' ? What can I do so that my after_request hook can still access the same HTTPResponse object?
At least in the master branch, a returned HTTPResponse will be
applied to the thread-local `bottle.request` before the
`after_request` hook is fired, so you should be able to see and
modify this response in said hook.
By 'applied' I mean that all state from the returned HTTPResponse is copied to the global `bottle.request` instance, overwriting any existing state. This is the desired behavior, as mixing these two would not make much sense.
I can think of 2 ways:
1) Instead of returning HTTPResponse, I just return a dictionary as the body: return json.dumps({'key','value'}). But how can I return a status code other than 200 now?
You can set `bottle.response.status = 200` or any other value and
then just return a dict. Bottle automatically encodes dicts to
json and sets the appropriate content-type header as well.
2) bottle.response.status = 200; bottle.response.body = json.dumps({'key':'value'}); return bottle.response instead.
This would not work as expected, because `bottle.response` is not a subclass of `HTTPResponse`. Both `LocalResponse` and `HTTPResponse` inherit from `BaseResponse`, but only `HTTPResponse` is handled as a special return type.
As mentioned earlier, do not mix `bottle.response` and returning
a `HTTPResponse`, because the latter will overwrite any changes
made to the former.
Is 2) thread-safe? Will it cause issues for multiple requests hitting the server and the returned responses get mixed up in the after_request hook?
`bottle.response` (an instance of `LocalResponse` is thread-save
(based on thread.local) and returning a `HTTPResponse` is also
thread-save.
Joe Zhang
At least in the master branch, a returned HTTPResponse will be applied to the thread-local `bottle.request` before the `after_request` hook is fired, so you should be able to see and modify this response in said hook.
By 'applied' I mean that all state from the returned HTTPResponse is copied to the global `bottle.request` instance, overwriting any existing state. This is the desired behavior, as mixing these two would not make much sense.
If you agree that the above is expected (the returned HTTPResponse will NOT be the same as what's referenced in bottle.response in the after_request hook), what is your recommended remedy to allow me to successfully set the headers on the response in the after_request hook?
Marcel Hellkamp
At least in the master branch, a returned HTTPResponse will be applied to the thread-local `bottle.request` before the `after_request` hook is fired, so you should be able to see and modify this response in said hook.
Was that a typo? Do you mean 'will be applied to the thread-locdal 'bottle.response' ? After all, the headers I'm trying to set in the after_request hook is on bottle.response
Yes that was a typo.
By 'applied' I mean that all state from the returned HTTPResponse is copied to the global `bottle.request` instance, overwriting any existing state. This is the desired behavior, as mixing these two would not make much sense.Are you implying that returning HTTPResponse, and letting the after_request hook set the header via bottle.response will work?
Yes, but as I said, only in the master branch.
The returned HTTPResponse is applied here:
https://github.com/bottlepy/bottle/blob/master/bottle.py#L1009
The after_request hook is called two lines after that.
The apply method is defined here: https://github.com/bottlepy/bottle/blob/master/bottle.py#L1953
But that's not what I observe. What I observe is that: if I set up a breakpoint on where bottle.response is referenced in after_hook, its 'body' attribtue contains '' (default), while my HTTPResponse had a non-empty body set. This implies to me that the HTTPResponse object was not being referenced to in the after_request hook.
I'd assume you are using bottle-0.12 then.
If you agree that the above is expected (the returned HTTPResponse will NOT be the same as what's referenced in bottle.response in the after_request hook), what is your recommended remedy to allow me to successfully set the headers on the response in the after_request hook?
In bottle-0.13 (not released) it just works. In bottle-0.12 I'd recommend to not use the after_request hook, but instead use a plugin to do the same: https://bottlepy.org/docs/0.12/plugindev.html
These work like decorators, but are applied to all route callbacks on demand. They have much greater control over return values, exception handling and other aspects than hooks would have. They even might be a little bit faster.
In your plugin, you can check yourself if the return value of the
route callback is a HTTPResponse and add the headers there, or
fall back to `bottle.response` if not.
Hope that helps.
Joe Zhang
Thank you for the response. I will investigate using a plugin.
In the meantime, I do have an some what related question:
I have created 2 different ways of returning from one of my route-decorated functions:
Method 1:
return HTTPResponse(status=200, body=myBody)
Method 2 (This is done in order to for after_request hook to read the same response object, for now.):
response = bottle.response.copy()
response.status = 200
response.body = myBody
return response
myBody is a string, NOT a dict (our upstream library already converts the dict into a string via json.dumps()), and myBody is also large: it is a dictionary with 1 key, but a 600+ member list as the value.
My problem is:
Method 1 returns very fast to the calling client (client is using requests library via a POST request).
Method 2 is about 10 times slower (the calling client waits about 2 mins to get the response, versus 2 seconds in method 1).
I've also disabled any after_hook logic just to isolate any other effects.
Any hint as to what might be the root cause?