POST request from within the app running on IOloop

[Dhiraj]

I have a tricky situation here.
I have app A and B running on a tornado server listening on  port 9090.
I have a method xxx defined in app A which internally attempts to make a POST request to some REST api zzz served by app B.
Surprisingly it works for any urls other than ones using port 9090.
I have tried using httpclient from tornado, urllib2, httplib, python requests lib.
All of them (except urllib) work when I call on other port but not the same port.
I am running similar config on another machine and I can make my method xxx call to zzz served on other machine.

Is it because of some IOloop feature, that I am not able to call from the server to same port behind which server itself is running?
What could be the workarounds??
Basically I have to do all this because I want to share some data between two apps A and B and dont want to use databases for it.
 

[Dhiraj]

I will explain the situation briefly.
I have two apps running on tornado server, one of them is simple tornado app and other one is kept in WSGIcontainer. I have put them on single ioloop instance. They are listening on same port 9090.

What I want to do is, when user calls some method on simple tornado app, the app will internally call some REST api served by other app inside WSGIcontainer.
But from my simple tornado app I am able to call any other url apart from what is served on my server and port.

I have tried multiple httpclient libraries.
Is there any fundamental mistake?

[Kevin LaTona]

Can you post some parts of your code that shows what you are doing and you feel like it might be the sticking point?

I've never had a need to use Tornado's wsgi connector, so maybe that is tripping you some how here.

If you all you need to do is listen on port 9090 and have that connection make a POST request to another app or server this is possible in Tornado.

Personally I see no real reason to use wsgi to do that… unless I am missing some reason why one would want to.

I would look at using Tornado's AsyncHttpClient to make your call out from the incoming GET call.

https://github.com/tornadoweb/tornado/blob/master/tornado/httpclient.py#L106

Past that I can see no reason why when using port 9090 would not work as a single end point vs why say port 8080 would.

-Kevin

-- 
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python-tornad...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dhiraj]

Yeah. This is the code.
First I am running app A and B.  App B uses flask libraries that is why I had to put it under wsgicontainer.

                         from tornado.ioloop import IOLoop
                         import app_A
                         import flask_app_B
                         def shutdown_hook():
                            IOLoop.instance().stop()
                         app_B = WSGIContainer(flask_app_B)
                         server_app = Application([(r"/random_dir", app_A),(r".*", FallbackHandler, dict(fallback=app_B))])
                         http_server = HTTPServer(server_app)
                         http_server.listen(9090)
                         IOLoop.instance().start()

Now inside my app_A, I have a method defined which calls a REST API served by app_B. My original purpose is to share some data between these two apps. I dont want go for database options as it will be an overkill solution. So I was hoping that anyway app_B has REST apis which could be called from inside app_A using httpclient libraries(or one you mentioned).
in that case typical work flow is

USER calls method1 inside app_A ----> method1 internally uses httpclient/adynchttpclient libraries to call REST api served by app_B --> Rest api from app_B gives reply to app_A's method1 ---> app_A's method1 replied back to user.

This is what i am doing inside when user calls method1 of app_A.

import httplib, time
import urlparse, tornado.httpclient

url = "http://localhost:9090/some_rest_api"
headers = {'some_data': 'my_data'}

                        

req = tornado.httpclient.HTTPRequest(url, method='GET', headers=headers)
http_client = tornado.httpclient.HTTPClient()
  try:
     response = http_client.fetch(req)
     print response.body
  except tornado.httpclient.HTTPError as e:
     # HTTPError is raised for non-200 responses; the response
     # can be found in e.response.
     print("Error1: " + str(e))
  except Exception as e:
     # Other errors are possible, such as IOError.
     print("Error2: " + str(e))
http_client.close()

Above code seems to work for any url apart from one served on this server. If I start some example server on other port and ask method1 to call that it is working.

In app_B(flask app) I have following rest_api defined to which i am calling.

@app_B.route('/some_rest_api')
def some_rest_api():
    return 'true'

I tried running httpclient code separately, which works. It calls this some_rest_api service and gets the response.
But not from within the app_A.

Anyway I am going to try run these two apps under separate ports on the same tornado instance.

On Thursday, January 7, 2016 at 5:50:34 PM UTC+5:30, Dhiraj wrote:

[Dhiraj]

I also tried putting these apps behind separate ports but still doesn't work.
seems like ioloop, wsgicontainer, application modules have some roll in this problem.

[Kevin LaTona]

If Tornado and Flask are both on the same ip address.

And Tornado is listening on port 9090

What port number is the Flask App listening on?

-Kevin

[Ben Darnell]

On Fri, Jan 8, 2016 at 4:38 AM, Dhiraj <dhira...@gmail.com> wrote:

This is what i am doing inside when user calls method1 of app_A.

import httplib, time
import urlparse, tornado.httpclient

url = "http://localhost:9090/some_rest_api"
headers = {'some_data': 'my_data'}

                        

req = tornado.httpclient.HTTPRequest(url, method='GET', headers=headers)
http_client = tornado.httpclient.HTTPClient()

You should not use the synchronous HTTPClient in a Tornado handler. This blocks the entire IOLoop until the request is complete, and in this case it results in a deadlock because while the IOLoop is blocked, it can't serve any other requests. You have to use AsyncHTTPClient to make HTTP requests without blocking. This is a good idea in all cases, but it is absolutely necessary when the app wants to make a request to itself.

From the flask (app B) side, you can't use AsyncHTTPClient, so if app B needs to make HTTP requests it will have to use a synchronous HTTPClient. This means that the flask app will not be able to make requests to either app A or B. If you need this capability you'll need to run app B on a multithreaded WSGI server (like uwsgi or gunicorn) instead of Tornado's WSGIContainer. 

-Ben 

--

[Dhiraj]

Thanks a lot Ben,
That is a great point.
It didn't cross my mind until yesterday. I am blocking the whole ioloop thread which is in fact not giving my app_B any time to respond. I will go with async httpclient and post the results.

@kevin
Both apps are listening on same port. They are running on single server listening on port 9090.