echoserver example - how to transfer (Reply) data back to client/browser

[sanjeev kumar]

Cody,
To get the echoserver example run the following lines were commented
out
/*
#ifndef WINDOWS
UnixAddress echoaddress("/tmp/echo");
Socket::ptr s = echoaddress.createSocket(ioManager, SOCK_STREAM);
s->bind(echoaddress);
Scheduler::getThis()->schedule(boost::bind(&socketServer, s));
#endif
*/
Now I do not get the Address in use.

Next, I have been trying to send data back data based on the query.
For example the echoserver should reply back with the file asked in
the URI.
http://http://10.1.113.90:43009/MyData/Myfile.txt

I do not know how to obtain the Myfile.txt from the query string.
Then what part of the response structure should be filled.

//This is how I am trying
void httpRequest(HTTP::ServerRequest::ptr request) {

const std::string &method = request->request().requestLine.method;
const URI &uri = request->request().requestLine.uri;
MORDOR_ASSERT(!uri.schemeDefined() || uri.scheme() == "http" ||
uri.scheme() == "https");

URI::Path requestPath = uri.path;
//requestPath.append("index.html");
std::string request_path = requestPath.toString();
std::cout << "RequestPath = " << request_path.c_str() << std::endl;


if (method == HTTP::GET || method == HTTP::HEAD) { //method ==
HTTP::PUT || method == HTTP::POST)
std::cout << "Inside HTTP::GET || method == HTTP::HEAD :::::" <<
std::endl;

//Fill the initial response data.
request->response().entity.contentLength =
request->request().entity.contentLength;
request->response().entity.contentType = request-
>request().entity.contentType;
request->response().general.transferEncoding = request-
>request().general.transferEncoding;
request->response().status.status = HTTP::OK;
request->response().entity.extension = request-
>request().entity.extension; //Map of extension headers.
if (uri == "/MyData/Myfile.txt") {
std::cout << "Inside MyFile Request" << std::endl;
// just open the file with name myfile.txt to send back.

std::ifstream iFile("MyFile.txt", std::ios::in |
std::ios::binary);
if (!iFile){
rep = reply::stock_reply(reply::not_found);
return;
}
char buf[512];
while (iFile.read(buf,
sizeof(buf)).gcount() > 0)
request-
>response().contentEncoding.push:bach(buf, iFile.gcount());
//how to store the file content in the Reposne structure.
//Is the respond stream is the way to
reply to the client.
respondStream(request, request->requestStream());

}
if (request->hasRequestBody()) {
if (request->request().requestLine.method != HTTP::HEAD) {
//a helper function, that u give the request object and a stream
and it will set up the header and take care of range request
std::cout << "Inside HTTP::GET || method == HTTP::HEAD-33334" <<
std::endl;
respondStream(request, request->requestStream());
return;

} else {
request->finish(); //if it was a head we finish.
}
} else { //if it didn't had a request body we say content length is
0
std::cout << "Inside HTTP::GET || method == HTTP::HEAD-44444" <<
std::endl;
request->response().entity.contentLength = 0;
request->finish();
}
} else {
respondError(request, HTTP::METHOD_NOT_ALLOWED); //other wise not
allowd.
}
std::cout << "Inside httpRequest- Completed" << std::endl;
//MORDOR_LOG_INFO("Inside httpRequest- Completed");
}

thank you for your insight.

[Cody Cutrer]

Sure thing.

#include "mordor/streams/file.h"

...

if (uri == "/MyData/Myfile.txt") {

FileStream::ptr fileStream(new FileStream("MyFile.txt"));
return respondStream(request, fileStream);
}

The key point is that you need to use a Mordor::Stream, not a
std::istream. respondStream will handle setting all of the HTTP
headers having to do with transferring the raw data (Content-Length,
Transfer-Encoding, doing Range requests, etc.), or doing a HEAD
instead of a GET; see . You can set additional headers if you would
like (Content-Type, E-Tag, Last-Modified-At).

One thing to be careful of is that file I/O is *not* asynchronous, so
while it is reading data from the file, it is blocking the current
thread, and not able to process concurrent requests. The easy way to
fix that is to create a WorkerPool of threads to handle file I/O, and
pass it to FileStream as the scheduler argument; then it will
automatically run any blocking calls on the worker threads, and not
block the main IOManager thread(s).

Cody

[sanjeev kumar]

Thanks.

Yes the I/O operation is clear that it needs to be performed
asynchronously. This I will do in the next steps.Once I am through
with basic steps.

meanwhile I get exception:

if (uri == "/MyData/MyFile.txt") {
// just open the file with name Myfile.txt to send back.

FileStream::ptr fileStream(new FileStream("/MyData/MyFile.txt",
FileStream::READ));

return respondStream(request, fileStream);

std::cout << "respondStream- Done" << std::endl;
}

I created a directory MyData and the file MyFile.txt under it. The
directory is created in the location from where I am launching the
echoserver.
By default where does the server looks for this path /MyData/
MyFile.txt. I was expecting that it will first look from the place
where I launch the server. ?

RequestPath = /MyData/MyFile.txt
2011-Sep-20 16:02:25.263026 3197315 ERROR 14796 0x89a2f0
mordor:http:server mordor/http/server.cpp:616 0x899cd0-1 Unexpected
exception: mordor/streams/file.cpp(84): Throw in function void
Mordor::FileStream::init(const std::string&,
Mordor::FileStream::AccessFlags, Mordor::FileStream::CreateFlags,
Mordor::IOManager*, Mordor::Scheduler*)
Dynamic exception type:
boost::exception_detail::clone_impl<Mordor::FileNotFoundException>
std::exception::what: std::exception
[Mordor::tag_backtrace*] = /home/b021272/workspace/lib/linux/
libmordor.so.
1(_ZN6Mordor10FileStream4initERKSsNS0_11AccessFlagsENS0_11CreateFlagsEPNS_9IOManagerEPNS_9SchedulerE
+0xd02) [0x7f157fc38332]
./MorderHTTPServer() [0x416114]
./
MorderHTTPServer(_Z11httpRequestN5boost10shared_ptrIN6Mordor4HTTP13ServerRequestEEE
+0x9f9) [0x412867]
./MorderHTTPServer() [0x41f866]
/home/b021272/workspace/lib/linux/libmordor.so.
1(_ZN6Mordor4HTTP13ServerRequest9doRequestEv+0x13e5) [0x7f157fb9b8b5]
/home/b021272/workspace/lib/linux/libmordor.so.
1(_ZN6Mordor5Fiber10entryPointEv+0x338) [0x7f157fa913e8]
/lib64/libc.so.6() [0x3e02c43ca0]
[boost::errinfo_errno_*] = 2, "No such file or directory"
[boost::errinfo_file_name_*] = /MyData/MyFile.txt
[boost::errinfo_api_function_*] = open

Thank you for your hint.

[Cody Cutrer]

It's looking for /MyData/MyFile.txt (in the root). You probably want
"MyData/MyFile.txt" to be relative to the current working directory.

Cody

[sanjeev kumar]

Hi Cody,

Continuing on the same.. After introducing the following in the run
method.
int run(int argc, char *argv[]) {

try {
IOManager ioManager;
startSocketServer(ioManager);

HTTP::ServletDispatcher dispatcher;
HTTP::Servlet::ptr files(new FileServlet), instrs(new
InstrumentServlet);

dispatcher.registerServlet("/", files);
dispatcher.registerServlet("instrument/", instrs); //
uri = id/name
dispatcher.registerServlet("order/", instrs); //
uri = id/name?lastSeen=20
dispatcher.registerServlet("quote/", instrs); //
uri = id/name?lastSeen=10
dispatcher.registerServlet("log/", instrs); //
uri = lastSeen=?

startHttpServer(ioManager);

ioManager.dispatch();
} catch (...) {
std::cerr <<
boost::current_exception_diagnostic_information()
<< std::endl;
return 1;
}
return 0;
}

Doing the above does not invoke the FileServlet nor the
InstrumentServlet, ... how can this be done

The aim of the FileServlet is to return all the files under / or from
other specified directory. I would expect that the below FileServlet
would return all the files names ?

namespace {
class FileServlet : public HTTP::Servlet
{
public:
void request(boost::shared_ptr<HTTP::ServerRequest>

request) {
const std::string &method = request-
>request().requestLine.method;

if (method == HTTP::GET) {

const URI &uri = request-
>request().requestLine.uri;

std::cout << "uri " << uri <<

std::endl;

request-
>response().entity.contentLength =

request-
>request().entity.contentLength;
request->response().entity.contentType
= request->request().entity.contentType;
request-
>response().general.transferEncoding = request-
>request().general.transferEncoding;
request->response().status.status =
HTTP::OK;
request->response().entity.extension =
request->request().entity.extension; //Map of extension headers.

if (uri == "index.html") {

// just open the file with
name Myfile.txt to send back.

FileStream::ptr fileStream(new

FileStream("http/" + uri.toString(), FileStream::READ));

return respondStream(request,
fileStream);
}
}
}
};

class InstrumentServlet : public HTTP::Servlet
{
public:
void request(boost::shared_ptr<HTTP::ServerRequest>

request) {
const std::string &method = request-
>request().requestLine.method;

if (method == HTTP::GET) {

const URI &uri = request-
>request().requestLine.uri;

request-
>response().entity.contentLength =
request-
>request().entity.contentLength;
request->response().entity.contentType
= request->request().entity.contentType;
request-
>response().general.transferEncoding = request-
>request().general.transferEncoding;
request->response().status.status =
HTTP::OK;
request->response().entity.extension =
request->request().entity.extension; //Map of extension headers.

std::cout << "uri " << uri <<
std::endl;


FileStream::ptr fileStream(new
FileStream("/instrument/" + uri.toString(), FileStream::READ));
return respondStream(request,
fileStream);
}
}
};
}

Thanks for your input.

Sanjeev

[Cody Cutrer]

You need to register your servlets with a leading slash
("/instruments/"). Also, the dispatcher does not modify the uri when
it dispatches, so inside your servlet you will still have the full URI
("/instruments/myfile.txt"). Be careful, though... someone could use
a URI like "/instruments/../../../etc/passwd" to try and read
sensitive data from your server.

Cody

[Cody Cutrer]

Btw, I should probably also mention that you don't need to do all that
work of copying headers for your own servlets; Echoserver only does
that because it's sending back in the response most stuff that it got
in the request.

On Tue, Sep 20, 2011 at 1:16 PM, sanjeev kumar <sanje...@gmail.com> wrote:

[sanjeev kumar]

I do register the servlets now with a leading slash, but still I do
not see any info in the log that the servlets are invoked ?

HTTP::ServletDispatcher dispatcher;
HTTP::Servlet::ptr files(new FileServlet), instrs(new
InstrumentServlet);
dispatcher.registerServlet("/", files);

dispatcher.registerServlet("/instrument/", instrs);

Sanjeev

[Cody Cutrer]

Case 1: No, FileStream cannot be used to output a directory listing.
While you may be able to open a directory, the data will be a raw
stream of directory entries, not anything that would be useful to a
user.
Case 2: To serve an entire directory, just do a partial match on the
URI, instead of an exact match like you're currently doing. But
again, beware of attacks to read files outside of the intended
directory.

Cody

On Wed, Sep 21, 2011 at 6:36 AM, sanjeev kumar <sanje...@gmail.com> wrote:
> other questions
>
> Case 1: If the uri == /html  (where html is directory.  )
> then wont all the files under this directory will be send via
> respondStream to the browser ?
> Case 2: If the uri == /html/index.html
> if (uri == "/html/index.html") {
>        std::string cPath = boost::filesystem::current_path().string();
>       std::cout << cPath.c_str() << std::endl;
>        FileStream::ptr fileStream(
>        new FileStream(cPath + uri.toString(), FileStream::READ));
>   return respondStream(request, fileStream);
> }
> Then index.html is send and displayed in the browser. Now how to serve
> other links which are in index.html. (i.e the navigation, how should
> the server handle it.
>
> Thank you.

[Cody Cutrer]

That's still a full equality check. You should do something like
if(uri.path.segments.size() > 2 && uri.path.segments[1] == 'html') -
only matching against the first two segments (the first being an empty
segment that makes the path absolute).

Cody

On Wed, Sep 21, 2011 at 9:32 AM, sanjeev kumar <sanje...@gmail.com> wrote:
> Thank you Cody,
> For Case 2 :
> So you mean I should do it in this way in the httpRequest method.
> if (uri == "/html/") {

>         std::string cPath =
> boost::filesystem::current_path().string();
>        std::cout << cPath.c_str() << std::endl;
>         FileStream::ptr fileStream(
>        new FileStream(cPath + uri.toString(), FileStream::READ));
>    return respondStream(request, fileStream);
>  }
>

> Also why are the servlets not getting invoked ?

> I do register the servlets now with a leading slash, but still I do
> not see any info in the log that the servlets are invoked ?

>       ......

>        HTTP::ServletDispatcher dispatcher;
>        HTTP::Servlet::ptr files(new FileServlet), instrs(new
> InstrumentServlet);
>        dispatcher.registerServlet("/", files);
>        dispatcher.registerServlet("/instrument/", instrs);

>       startHttpServer(ioManager);
>     ioManager.dispatch();
>
> thank you
> Sanjeev

[sanjeev kumar]

thank you. that works!

WorkerPool:
I was also trying to put the I/O in asyn. mode as briefed by you
earlier.
How to associate/pass the workerPool to FileStream as the scheduler
argument.

This is what I do:
..
//startHttpServer(ioManager);
WorkerPool workerPool(3, false);
ioManager.schedule(boost::bind(&startHttpServer,
boost::ref(ioManager), boost::ref(workerPool)));

Then how to to pass the workerPool as schedular to the httpRequest
method which is called from httpServer as FileStream is in httpRequest
method.

Thank you.

[Cody Cutrer]

Pass the WorkerPool as a constructor argument to FileServlet.

[sanjeev kumar]

But none of the Servlets are triggered, even after registering it.

Why are the servlets not getting invoked ?

I do register the servlets now with a leading slash, but still I do
not see any info in the log that the servlets are invoked ?
......
HTTP::ServletDispatcher dispatcher;
HTTP::Servlet::ptr files(new FileServlet), instrs(new
InstrumentServlet);
dispatcher.registerServlet("/", files);
dispatcher.registerServlet("/instrument/", instrs);
startHttpServer(ioManager);
ioManager.dispatch();

Thank you.

[sanjeev kumar]

Cody,

In fact I was trying this way to put the dispatcher in the request
processing path.

Scheduler::getThis()->schedule(
boost::bind(&HTTP::ServletDispatcher::request, files));
but this doesn't work.
Could you please provide a sample how trigger the servlets.

Sanjeev

[Cody Cutrer]

It doesn't look like you passed the dispatcher to your
HTTP::ServerConnection. Just pass it through startHttpServer, and
then when you create the connection do new
HTTP::ServerConnection(stream, dispatcher) (no bind necessary).

Cody

[sanjeev kumar]

Hi Cody,

Could you please give an example how to pass dispatcher.
I tried but it is not working.
--------------

FileStream("http/" + uri.toString(), FileStream::READ));

return respondStream(request,
fileStream);
}
}
}
};
}
void httpServer(Socket::ptr listen, HTTP::ServletDispatcher
&dispatcher) {
listen->listen();
//MORDOR_LOG_INFO("Inside httpServer");
std::cout << "Inside httpServer" << std::endl;
while (true) {
Socket::ptr socket = listen->accept();
Stream::ptr stream(new SocketStream(socket));
HTTP::ServerConnection::ptr conn(
new HTTP::ServerConnection(stream,
dispatcher)); //<<<<-----
Scheduler::getThis()->schedule(

boost::bind(&HTTP::ServerConnection::processRequests, conn));
}
std::cout << "Inside httpServer- Completed" << std::endl;
//MORDOR_LOG_INFO("Inside httpServer- Completed");
}

void startHttpServer(IOManager &ioManager, HTTP::ServletDispatcher&
dispatcher) {
std::vector < Address::ptr > addresses = Address::lookup(
"10.1.113.90:43009", AF_UNSPEC, SOCK_STREAM);

std::vector<Address::ptr>::const_iterator
it(addresses.begin());
MORDOR_LOG_INFO(Log::root()) << *it;
//for (std::vector<Address::ptr>::const_iterator
it(addresses.begin());
for (; it != addresses.end(); ++it) {
Socket::ptr s = (*it)->createSocket(ioManager,
SOCK_STREAM);
s->bind(*it);
Scheduler::getThis()-
>schedule(boost::bind(&httpServer, s, dispatcher)); <<-----

}

}

int run(int argc, char *argv[]) {

try {
IOManager ioManager;
startSocketServer(ioManager);

HTTP::ServletDispatcher dispatcher;
HTTP::Servlet::ptr files(new FileServlet), instrs(new
InstrumentServlet);

dispatcher.registerServlet("/", files);

dispatcher.registerServlet("instrument/", instrs);
dispatcher.registerServlet("order/", instrs);
dispatcher.registerServlet("quote/", instrs);
dispatcher.registerServlet("log/", instrs);

startHttpServer(ioManager, dispatcher);

ioManager.dispatch();
} catch (...) {
std::cerr <<
boost::current_exception_diagnostic_information()
<< std::endl;
return 1;
}
return 0;
}

-------------
Thank you
Sanjeev

[Cody Cutrer]

When passing ServletDispatcher to httpServer in the boost::bind,
you'll need to wrap it in boost::ref, like so:

Scheduler::getThis()->schedule(boost::bind(&httpServer, s,

boost::ref(dispatcher));

Cody

[sanjeev kumar]

Thanks.

Doing this the dispatche invokes only FileServlet other servlets are
not invoked ? how to invoke the remaning servlets.


And other thing, I was trying to pass the worker pool to the
FileServlet via contructor :

-->FileServlet(IOManager ioMan, WorkerPool
wPool) :ioManager(ioMan),workers(wPool) {}
...

FileStream::ptr fileStream( new FileStream(cPath + uri.toString(),

FileStream::READ, ioManager, workers)); <<-- this complains

.......
.....
int main() {

WorkerPool workerPool(3, false);
ioManager.schedule(boost::bind(&startHttpServer,

boost::ref(ioManager), boost::ref(workerPool))); <<-- this complains
HTTP::ServletDispatcher dispatcher;
HTTP::Servlet::ptr files(new FileServlet(ioManager, workerPool)),

instrs(new InstrumentServlet);
dispatcher.registerServlet("/", files);
dispatcher.registerServlet("/instrument/", instrs);

dispatcher.registerServlet("/order/", instrs);
dispatcher.registerServlet("/quote/", instrs);
dispatcher.registerServlet("/log/", instrs);

startHttpServer(ioManager, dispatcher);
}
ioManager.dispatch();

Thank you
Sanjeev

> ...
>
> Erfahren Sie mehr »

[Cody Cutrer]

Schedulers (IOManager and WorkerPool) are non-copyable. You need to
either pass them by reference, or pass a pointer to them.

What URLs are you using to access the other servlets? Because you
passed the trailing slash when registering them, you need to access
them with a trailing slash (i.e. /instrument will get routed to the
files controller registered for /, because it does not match
/instrument/).

Cody