[Caml-list] ocamlnet mini tutorial: netclient and cookie based authentication
Hendrik Tews
for ocamlnet novices I document here what you need to access web
sites that use cookies for authentication (eg, wikis or mailman).
With this you can let ocaml perform your mailing-list
administrator tasks or write a wiki bot.
Cookie authentication works as follows: When you first visit the
site you have to authenticate with username and password that are
usually sent (in cleartext) in a post request to the server. The
server responses with a cookie that will be used for
authentication in all following requests.
Because you have to retrieve and set cookies in the headers you
cannot use the Http_client.Convenience module. You really have to
dive into ocamlnet.
Here comes the solution:
Netstring extension:
- You need get_set_cookie, which will retrieve cookies from a
response header but which is not in ocamlnets netstring library
currently. Download from [1] or with two additional helper
functions from [2].
Prerequisites:
- Create a pipeline with "new Http_client.pipeline", it will
process all http requests. (Don't ask me about timeouts and
retries, the defaults work fine for me.)
Get going (download some page without authentication):
- Create a get call "new Http_client.get url"
- Process the call: Add it to the pipeline (method #add) and run
the pipeline (#run). By this the call will be changed and
contain the response.
- Check the response status (#response_status), which should
usually be `OK and get the page (#response_body#value).
- In case of error convert the status into something readable
with Nethttp.string_of_http_status or consult the response
status message (#response_status_text).
Login:
- Create post call with
"new Http_client.post url parameters", the parameters are a
string * string list, eq
[("wpName", "mywikiname");
("wpPassword", "mywikipasswd");
("wpLoginattempt", "Log in")]
for a MediaWiki.
[In order to get the parameters save the login page, change the
method of the right from into "GET", load the page into a
browser, press the login button and parse the new URL you get.]
- Process the call.
- Check the status (#response_status): For MediaWiki it should be
`Found, for mailman and others `Ok (don't ask me why).
- Retrieve the response header (#response_header) and extract the
cookies with get_set_cookies from [2].
Make an authenticated request:
- Create your call "new get url" for getting a page or
"new post url" for performing an action.
- Access the request header (#request_header `Base) and add your
cookies there (set_cookies from [2]). This is all done via
side-effects, no need to write the modified header back into
the call.
- Process the call and get the response.
Compilation:
- with findlib and -package netclient -linkpkg:
ocamlfind {ocamlc,ocamlopt} -package netclient -linkpkg your files ...
Acknowledgements:
- Praise Gerd Stolpmann for ocamlnet and findlib.
That's it. For an example go to [3], which contains a MediaWiki
bot that updates structural information in the Mozilla wiki [4].
The needed utility functions are in wiki_http: wiki_login to
login, download to download a web page, get_page to get a wiki page
and write_wiki_page to write one.
Resources:
[3] http://www.sos.cs.ru.nl/cgi-bin/~tews/olmar/viewvc-patch.cgi/elsa/olmar/wiki_bot/
[4] http://wiki.mozilla.org/Elsa_ast_nodes
Bye,
Hendrik
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs