[Caml-list] ocamlnet mini tutorial: netclient and cookie based authentication

Skip to first unread message

Hendrik Tews

Jan 24, 2008, 4:50:55 PM1/24/08
to caml...@inria.fr

for ocamlnet novices I document here what you need to access web
sites that use cookies for authentication (eg, wikis or mailman).
With this you can let ocaml perform your mailing-list
administrator tasks or write a wiki bot.

Cookie authentication works as follows: When you first visit the
site you have to authenticate with username and password that are
usually sent (in cleartext) in a post request to the server. The
server responses with a cookie that will be used for
authentication in all following requests.

Because you have to retrieve and set cookies in the headers you
cannot use the Http_client.Convenience module. You really have to
dive into ocamlnet.

Here comes the solution:

Netstring extension:

- You need get_set_cookie, which will retrieve cookies from a
response header but which is not in ocamlnets netstring library
currently. Download from [1] or with two additional helper
functions from [2].


- Create a pipeline with "new Http_client.pipeline", it will
process all http requests. (Don't ask me about timeouts and
retries, the defaults work fine for me.)

Get going (download some page without authentication):

- Create a get call "new Http_client.get url"

- Process the call: Add it to the pipeline (method #add) and run
the pipeline (#run). By this the call will be changed and
contain the response.

- Check the response status (#response_status), which should
usually be `OK and get the page (#response_body#value).

- In case of error convert the status into something readable
with Nethttp.string_of_http_status or consult the response
status message (#response_status_text).


- Create post call with
"new Http_client.post url parameters", the parameters are a
string * string list, eq
[("wpName", "mywikiname");
("wpPassword", "mywikipasswd");
("wpLoginattempt", "Log in")]
for a MediaWiki.

[In order to get the parameters save the login page, change the
method of the right from into "GET", load the page into a
browser, press the login button and parse the new URL you get.]

- Process the call.

- Check the status (#response_status): For MediaWiki it should be
`Found, for mailman and others `Ok (don't ask me why).

- Retrieve the response header (#response_header) and extract the
cookies with get_set_cookies from [2].

Make an authenticated request:

- Create your call "new get url" for getting a page or
"new post url" for performing an action.

- Access the request header (#request_header `Base) and add your
cookies there (set_cookies from [2]). This is all done via
side-effects, no need to write the modified header back into
the call.

- Process the call and get the response.


- with findlib and -package netclient -linkpkg:

ocamlfind {ocamlc,ocamlopt} -package netclient -linkpkg your files ...


- Praise Gerd Stolpmann for ocamlnet and findlib.

That's it. For an example go to [3], which contains a MediaWiki
bot that updates structural information in the Mozilla wiki [4].
The needed utility functions are in wiki_http: wiki_login to
login, download to download a web page, get_page to get a wiki page
and write_wiki_page to write one.


[1] https://godirepo.camlcity.org/wwwsvn/trunk/code/get-set-cookie.ml?rev=1145&root=lib-ocamlnet2&view=auto

[2] http://www.sos.cs.ru.nl/cgi-bin/~tews/olmar/viewvc-patch.cgi/elsa/olmar/wiki_bot/netstring_ext.ml?revision=HEAD&view=markup

[3] http://www.sos.cs.ru.nl/cgi-bin/~tews/olmar/viewvc-patch.cgi/elsa/olmar/wiki_bot/

[4] http://wiki.mozilla.org/Elsa_ast_nodes



Caml-list mailing list. Subscription management:
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

Reply all
Reply to author
0 new messages