mwlib - what works and does not under windows

[brion....@gmail.com]

This is a progress report on what I have been able to get working
under windows. It is very close, but my head is sore from beating it
against the wall all day.

I used python easy-install to install mwlib-0.8.0 and mwlib.rl-0.8.0.
I have some notes on how I did this, but I will have to go back
through to remember what they mean and then I can share that process
here. That part wasn't too bad.

I have installed:
mwlib-0.8.0
mwlib.rl-0.8.0
Pygments-0.10
pyparsing-1.5.0
reportlab (svn version)

wiki setup:
* Mediawiki 1.12
* Collection extension - tarball downloaded from Mediawiki.org
7/20/2008
* read restricted access, unauthenticated users cannot view web pages:
** that is: $wgGroupPermissions['*' ]['read'] = false;

I have two methods that almost work.

Method 1:
Setup:
* Using pdfserver.py from mwlib-0.7.0
** that is: $wgCollectionMWServeURL = 'http://servername/cgi-bin/
pdfserver.py';
* modifying wiki setup to allow unauthenticated users to view web
** that is: $wgGroupPermissions['*' ]['read'] = true;
* modifying recorddb.py from mwlib for cross-platform issue:
** line 333-ish: os.rename(output + '.tmp', output)
** changeto:
os.unlink(output)
os.rename(output + '.tmp', output)
(python maps to the OS's rename, windows will not overwrite with
rename like Unix will.)

Result:
I can succesfully generate PDF files on at least some pages. Images
are included.

The main problem here is that pdfserver.py does not support
authentication, so I have been trying to move to mwlib.cgi which
combined with other changes in mwlib-0.8.0 posted about here ought to
be able to handle authentication. Which brings me to method 2.

Method 2:
Setup:
* Using mwlib.cgi from mwlib-0.8.0
** $wgCollectionMWServeURL = 'http://servername/cgi-bin/mwlib.cgi';
* adding configuration information for authentication
** $wgCollectionMWServeCredentials = "mwlib-user:mwlib-password";
* modifying wiki setup to allow unauthenticated users to view web
(because I am still having authentication issues)
** that is: $wgGroupPermissions['*' ]['read'] = true;
* modifying the following files for cross-platform issues:
** wsgi.py: ~ line 50 there is a content length mismatch. The byte
count difference implies that it might be due to line termination
characters. I have not yet verified, since I couldn't see a way to
easily do that. After verifying that the return data looked good, I
commented out the block of code that aborted on length mismatch.
** serve.py: ~ line 30 change 'subprocess.Popen(args, close_fds=True)'
to 'subprocess.Popen(args, close_fds=False)'. Windows does not
support close_fds=True.
** xhtmlwriter.py: ~ line 727 change 'p =subprocess.Popen(cmd,
shell=True,stderr=subprocess.PIPE, close_fds=True)' to 'p
=subprocess.Popen(cmd, shell=True,stderr=subprocess.PIPE,
close_fds=False)'. Windows does not support cloe_fds=True.
** recorddb.py: ~ line 333 - with this method, "collection.zip" does
not exist prior to the os.rename operation, so the os.unlink that I
added in method 1 above does not work and has to be removed. I tried
wrapping this with a try block but that did not work. I'll revisit
that later.

Results:
- there is still an issue with authentication. I might be able to get
some more info from the mediawiki log files. For now, I disabled
authentication.
- When generating the PDF file - the PDF is blank. Unlike method 1
which generated a PDF file with content, method 2 gave me a blank
document.


That's all I've got for now. Any hints about where to look next? The
next questions I am trying to answer are: Why is the PDF blank on
method 2? Why isn't authentication working on method 2?

[Ralf Schmitt]

brion....@gmail.com wrote:

> ** line 333-ish: os.rename(output + '.tmp', output)
> ** changeto:
> os.unlink(output)
> os.rename(output + '.tmp', output)

You should wrap that os.unlink in a try/except statement.

> (python maps to the OS's rename, windows will not overwrite with
> rename like Unix will.)

yes, windows sucks.

[Johannes Beigel]

Am 21.07.2008 um 08:35 schrieb brion....@gmail.com:
> * modifying recorddb.py from mwlib for cross-platform issue:
> ** line 333-ish: os.rename(output + '.tmp', output)
> ** changeto:
> os.unlink(output)
> os.rename(output + '.tmp', output)

I added this (with an if os.path.exists()) in the hg repository.

> ** $wgCollectionMWServeCredentials = "mwlib-user:mwlib-password";
> * modifying wiki setup to allow unauthenticated users to view web
> (because I am still having authentication issues)

Could you tell us a little bit more about the problems? Log output etc?

> * modifying the following files for cross-platform issues:
> ** wsgi.py: ~ line 50 there is a content length mismatch. The byte
> count difference implies that it might be due to line termination
> characters.

Could be the same problem we had in http://code.pediapress.com/wiki/ticket/191

I hoped that the new WSGI code would handle this automatically. Seems
like it doesn't. I'll look after this issue: http://code.pediapress.com/wiki/ticket/228

> ** serve.py: ~ line 30 change 'subprocess.Popen(args, close_fds=True)'
> to 'subprocess.Popen(args, close_fds=False)'. Windows does not
> support close_fds=True.

Adjusted in hg repository.

> - When generating the PDF file - the PDF is blank. Unlike method 1
> which generated a PDF file with content, method 2 gave me a blank
> document.

Might be the same Content-Length/line-ending problem as above.

Regards,
Johannes Beigel

[Johannes Beigel]

Am 21.07.2008 um 14:13 schrieb Johannes Beigel:
> Am 21.07.2008 um 08:35 schrieb brion....@gmail.com:
>> ** wsgi.py: ~ line 50 there is a content length mismatch. The byte
>> count difference implies that it might be due to line termination
>> characters.
>
> Could be the same problem we had in http://code.pediapress.com/wiki/ticket/191

Can you check if this has been fixed with this changeset:

http://code.pediapress.com/hg/mwlib/rev/f4e0e88056af

?

Thanks and regards,
Johannes Beigel

[Brion]

Just a quick response, I haven't dug into the result much, but here is
what happened with the change applied, from the mwlib.cgi log:

Traceback (most recent call last):
File "C:/xampp/cgi-bin/mwlib.cgi", line 65, in <module>
queue_dir=QUEUE_DIR,
File "C:\Python25\lib\cgi.py", line 64, in run

File "C:\Python25\lib\site-packages\mwlib-0.8.0-py2.5-win32.egg\mwlib
\wsgi.py", line 91, in __call__
for fd in (env['wsgi.input'], env['wsgi.output']):
KeyError: 'wsgi.output'


On Jul 21, 7:26 am, Johannes Beigel <johannes.bei...@brainbot.com>
wrote:

> Am 21.07.2008 um 14:13 schrieb Johannes Beigel:
>

> > Am 21.07.2008 um 08:35 schrieb brion.fin...@gmail.com:
> >> ** wsgi.py: ~ line 50 there is a content length mismatch.  The byte
> >> count difference implies that it might be due to line termination
> >> characters.
>

> > Could be the same problem we had inhttp://code.pediapress.com/wiki/ticket/191

[Johannes Beigel]

Am 21.07.2008 um 17:04 schrieb Brion:
> here is what happened with the change applied, from the mwlib.cgi log:
>
> Traceback (most recent call last):
> File "C:/xampp/cgi-bin/mwlib.cgi", line 65, in <module>
> queue_dir=QUEUE_DIR,
> File "C:\Python25\lib\cgi.py", line 64, in run
>
> File "C:\Python25\lib\site-packages\mwlib-0.8.0-py2.5-win32.egg\mwlib
> \wsgi.py", line 91, in __call__
> for fd in (env['wsgi.input'], env['wsgi.output']):
> KeyError: 'wsgi.output'

Uh, ok. There is no wsgi.output, that should be correctly handled
internally by the WSGI implementation (hopefully it does; this
behavior is explicitly mentioned in the WSGI PEP 333
http://www.python.org/dev/peps/pep-0333/).

I fixed the code in the hg repository, so that only wsgi.input is set
to binary mode.

Regards,
Johannes

[Brion]

Confirmation that the fix works in resolving the content length
problem.

On Jul 21, 10:33 am, Johannes Beigel <johannes.bei...@brainbot.com>
wrote:

> Am 21.07.2008 um 17:04 schrieb Brion:
>
> > here is what happened with the change applied, from the mwlib.cgi log:
>
> > Traceback (most recent call last):
> > File "C:/xampp/cgi-bin/mwlib.cgi", line 65, in <module>
> > queue_dir=QUEUE_DIR,
> > File "C:\Python25\lib\cgi.py", line 64, in run
>
> > File "C:\Python25\lib\site-packages\mwlib-0.8.0-py2.5-win32.egg\mwlib
> > \wsgi.py", line 91, in __call__
> > for fd in (env['wsgi.input'], env['wsgi.output']):
> > KeyError: 'wsgi.output'
>
> Uh, ok. There is no wsgi.output, that should be correctly handled
> internally by the WSGI implementation (hopefully it does; this

> behavior is explicitly mentioned in the WSGI PEP 333http://www.python.org/dev/peps/pep-0333/).

[Brion]

Here is the apache log that goes along with the other logs.

I can see the action=query calls to mediawiki api.php, but I do not
see any action=login calls or any other user credentials.

192.168.0.57 - - [21/Jul/2008:23:35:13 -0500] "GET /wiki/index.php/
Special:Random HTTP/1.1" 302 20
192.168.0.57 - - [21/Jul/2008:23:35:13 -0500] "GET /wiki/index.php/
Company_Starting_Notes HTTP/1.1" 200 5479
192.168.0.57 - - [21/Jul/2008:23:35:41 -0500] "GET /wiki/index.php/
Special:Collection/render_article/?arttitle=Company_Starting_Notes
HTTP/1.1" 302 20
192.168.0.66 - - [21/Jul/2008:23:35:42 -0500] "POST /cgi-bin/mwlib.cgi
HTTP/1.1" 200 101
192.168.0.57 - - [21/Jul/2008:23:35:42 -0500] "GET /wiki/index.php/
Special:Collection/rendering/?return_to=Company+Starting
+Notes&collection_id=dd6455a9faa9f1d8&writer=rl HTTP/1.1" 200 3268
192.168.0.66 - - [21/Jul/2008:23:35:43 -0500] "GET /wiki/api.php?
action=query&meta=siteinfo&format=json HTTP/1.1" 200 57
192.168.0.66 - - [21/Jul/2008:23:35:41 -0500] "POST /cgi-bin/mwlib.cgi
HTTP/1.1" 200 53
192.168.0.66 - - [21/Jul/2008:23:35:45 -0500] "POST /cgi-bin/mwlib.cgi
HTTP/1.1" 200 101
192.168.0.57 - - [21/Jul/2008:23:35:45 -0500] "GET /wiki/index.php/
Special:Collection/rendering/?
collection_id=dd6455a9faa9f1d8&writer=rl&return_to=Company+Starting
+Notes HTTP/1.1" 200 3264
192.168.0.66 - - [21/Jul/2008:23:35:48 -0500] "POST /cgi-bin/mwlib.cgi
HTTP/1.1" 200 101
192.168.0.57 - - [21/Jul/2008:23:35:48 -0500] "GET /wiki/index.php/
Special:Collection/rendering/?
collection_id=dd6455a9faa9f1d8&writer=rl&return_to=Company+Starting
+Notes HTTP/1.1" 200 3264
192.168.0.57 - - [21/Jul/2008:23:35:48 -0500] "GET /wiki/skins/common/
shared.css?116 HTTP/1.1" 304 -
192.168.0.57 - - [21/Jul/2008:23:35:48 -0500] "GET /wiki/skins/
monobook/main.css?116 HTTP/1.1" 304 -
192.168.0.57 - - [21/Jul/2008:23:35:48 -0500] "GET /wiki/skins/common/
commonPrint.css?116 HTTP/1.1" 304 -
192.168.0.57 - - [21/Jul/2008:23:35:49 -0500] "GET /wiki/skins/common/
wikibits.js?116 HTTP/1.1" 304 -
192.168.0.57 - - [21/Jul/2008:23:35:49 -0500] "GET /wiki/skins/common/
ajax.js?116 HTTP/1.1" 304 -
192.168.0.57 - - [21/Jul/2008:23:35:49 -0500] "GET /wiki/skins/common/
ajaxwatch.js?116 HTTP/1.1" 304 -
192.168.0.57 - - [21/Jul/2008:23:35:49 -0500] "GET /wiki/index.php?
title=-&action=raw&smaxage=0&gen=js&useskin=monobook HTTP/1.1" 200 208
192.168.0.57 - - [21/Jul/2008:23:35:49 -0500] "GET /wiki/index.php?
title=MediaWiki:Common.css&usemsgcache=yes&action=raw&ctype=text/
css&smaxage=18000 HTTP/1.1" 200 67
192.168.0.57 - - [21/Jul/2008:23:35:49 -0500] "GET /wiki/index.php?
title=MediaWiki:Monobook.css&usemsgcache=yes&action=raw&ctype=text/
css&smaxage=18000 HTTP/1.1" 200 143
192.168.0.57 - - [21/Jul/2008:23:35:49 -0500] "GET /wiki/index.php?
title=-&action=raw&gen=css&maxage=18000&smaxage=0&ts=20080721223501
HTTP/1.1" 200 92
192.168.0.57 - - [21/Jul/2008:23:35:49 -0500] "GET /wiki/index.php?
title=User:Brion/monobook.css&action=raw&ctype=text/css HTTP/1.1" 200
31
192.168.0.57 - - [21/Jul/2008:23:35:49 -0500] "GET /wiki/index.php?
title=User:Brion/monobook.js&action=raw&ctype=text/
javascript&dontcountme=s HTTP/1.1" 200 241
192.168.0.57 - - [21/Jul/2008:23:35:50 -0500] "GET /wiki/skins/
monobook/headbg.jpg HTTP/1.1" 304 -
192.168.0.57 - - [21/Jul/2008:23:35:50 -0500] "GET /wiki/skins/common/
images/poweredby_mediawiki_88x31.png HTTP/1.1" 304 -
192.168.0.57 - - [21/Jul/2008:23:35:50 -0500] "GET /wiki/skins/
monobook/bullet.gif HTTP/1.1" 304 -
192.168.0.57 - - [21/Jul/2008:23:35:50 -0500] "GET /wiki/skins/
monobook/user.gif HTTP/1.1" 304 -
192.168.0.57 - - [21/Jul/2008:23:35:50 -0500] "GET /wiki/
corventis_logo_2.png HTTP/1.1" 304 -
192.168.0.66 - - [21/Jul/2008:23:35:52 -0500] "POST /cgi-bin/mwlib.cgi
HTTP/1.1" 200 101
192.168.0.57 - - [21/Jul/2008:23:35:52 -0500] "GET /wiki/index.php/
Special:Collection/rendering/?
collection_id=dd6455a9faa9f1d8&writer=rl&return_to=Company+Starting
+Notes HTTP/1.1" 200 3264



On Jul 22, 12:03 am, Brion <brion.fin...@gmail.com> wrote:
> A little bit more about the authentication problem.
>
> I have the following log files:
> 1. mwlib.cgi.log - log file from mwlib.cgi
> 2. mw-render.log - log file from mw-render
> 3. collection.log - a log file created by some debug statements that I
> added to the collection extension, which shows the json, URL, and
> arguments being posted to the mwlib.cgi
> 4. mediawiki log - log file created by mediawiki showing the requests
> that mediawiki sees.
>
> The mediawiki log looks the most interesting. The call to api.php
> does not seem to contain any credentials nor does it seem to use an
> action=login command to retrieve an authentication cookie.
>
> excerpt from mediawiki log:
> ---------------------------------------
> Start request
> GET /wiki/api.php?action=query&meta=siteinfo&format=json
> Accept-Encoding: identity
> Host: server
> Connection: close
> User-Agent: mwlib
>
> Main cache: FakeMemCachedClient
> Message cache: MediaWikiBagOStuff
> Parser cache: MediaWikiBagOStuff
> Unstubbing $wgParser on call of $wgParser->setFunctionHook from
> wfSetupPathFunctions
> Unstubbing $wgContLang on call of $wgContLang->getMagic from
> MagicWord::load
> Unstubbing $wgMessageCache on call of $wgMessageCache->addMessages
> from wfSetupPathFunctions
> Fully initialised
> Unstubbing $wgUser on call of $wgUser->isAllowed from
> ApiMain::__construct
> Profile section ended by close(): API:json
> Profile section ended by close(): API:main
> Profile section ended by close(): api.php
> Profile section ended by close(): -total
> 20080722043543 0.064 /wiki/api.php?
> action=query&meta=siteinfo&format=json anon
>

[Brion]

The rest of the logs follow in full, except that I trimmed profiler
information out of the mediawiki log and hopefully didn't trim too
much.

mwlib.cgi.log - this log file does not look particularly interesting
-------------------
mwlib.serve.info >> Creating new collection dir 'c:/xampp/var/cache/
pdfserver/dd6455a9faa9f1d8'
mwlib.serve.info >> render dd6455a9faa9f1d8 rl
mwlib.serve.info >> rendering 'c:/xampp/var/cache/pdfserver/
dd6455a9faa9f1d8\\output.rl'
mwlib.wsgi.info >> request took 0.016000 s
mwlib.serve.info >> render_status dd6455a9faa9f1d8 rl
mwlib.wsgi.info >> request took 0.000000 s
mwlib.serve.info >> render_status dd6455a9faa9f1d8 rl
mwlib.wsgi.info >> request took 0.000000 s
mwlib.serve.info >> render_status dd6455a9faa9f1d8 rl
mwlib.wsgi.info >> request took 0.000000 s
mwlib.serve.info >> render_status dd6455a9faa9f1d8 rl
mwlib.wsgi.info >> request took 0.000000 s

mw-render.log - this log file claims that the base URL is invalid, but
it only claims this when read access is restricted. When read access
is not restricted, it does not complain.
--------------------
mwlib.utils.info >> fetching 'http://server/wiki/api.php?
action=query&meta=siteinfo&format=json'
mwlib.utils.info >> got 'http://server/wiki/api.php?
action=query&meta=siteinfo&format=json' (57 Bytes)

Traceback (most recent call last):

File "c:\python25\scripts\mw-render-script.py", line 8, in <module>
load_entry_point('mwlib==0.8.0', 'console_scripts', 'mw-render')()
File "c:\python25\lib\site-packages\mwlib-0.8.0-py2.5-win32.egg\mwlib
\apps.py", line 249, in render
options, args = parser.parse_args()
File "c:\python25\lib\site-packages\mwlib-0.8.0-py2.5-win32.egg\mwlib
\options.py", line 97, in parse_args
self.env = self.makewiki()
File "c:\python25\lib\site-packages\mwlib-0.8.0-py2.5-win32.egg\mwlib
\options.py", line 101, in makewiki
env = wiki.makewiki(self.options.config, metabook=self.metabook)
File "c:\python25\lib\site-packages\mwlib-0.8.0-py2.5-win32.egg\mwlib
\wiki.py", line 245, in makewiki
res = _makewiki(conf, metabook)
File "c:\python25\lib\site-packages\mwlib-0.8.0-py2.5-win32.egg\mwlib
\wiki.py", line 202, in _makewiki
res.images = image_mwapi(url)
File "c:\python25\lib\site-packages\mwlib-0.8.0-py2.5-win32.egg\mwlib
\wiki.py", line 68, in image_mwapi
return mwapidb.ImageDB(base_url, username=username,
password=password)
File "c:\python25\lib\site-packages\mwlib-0.8.0-py2.5-win32.egg\mwlib
\mwapidb.py", line 226, in __init__
assert self.api_helper.is_usable(), 'invalid base URL %r' %
base_url
AssertionError: invalid base URL 'http://server/wiki'

Collection.log - everything looks good here
-------------------
2008-07-22 04:35:41 wikidb: args:
2008-07-22 04:35:41 wikidb: $args[metabook]:
{"type":"collection","licenses":
[{"type":"license","name":"License","mw_rights_icon":"","mw_rights_page":"","mw_rights_url":"","mw_rights_text":""}],"items":
[{"type":"article","content-type":"text\/x-wiki","title":"Company
Starting Notes","timestamp":1212005783}]}
2008-07-22 04:35:41 wikidb: $args[base_url]: http://server/wiki
2008-07-22 04:35:41 wikidb: $args[template_blacklist]:
MediaWiki:PDF Template Blacklist
2008-07-22 04:35:41 wikidb: $args[writer]: rl
2008-07-22 04:35:41 wikidb: $args[command]: render
2008-07-22 04:35:41 wikidb: $args[login_credentials]: mwlib-
user:mwlib-password
2008-07-22 04:35:41 wikidb: url: http://server.domain.name/cgi-bin/mwlib.cgi
2008-07-22 04:35:41 wikidb: postFields:
2008-07-22 04:35:41 wikidb: postFields[metabook]:
{"type":"collection","licenses":
[{"type":"license","name":"License","mw_rights_icon":"","mw_rights_page":"","mw_rights_url":"","mw_rights_text":""}],"items":
[{"type":"article","content-type":"text\/x-wiki","title":"Company
Starting Notes","timestamp":1212005783}]}
2008-07-22 04:35:41 wikidb: postFields[base_url]: http://server/wiki
2008-07-22 04:35:41 wikidb: postFields[template_blacklist]:
MediaWiki:PDF Template Blacklist
2008-07-22 04:35:41 wikidb: postFields[writer]: rl
2008-07-22 04:35:41 wikidb: postFields[command]: render
2008-07-22 04:35:41 wikidb: postFields[login_credentials]: mwlib-
user:mwlib-password
2008-07-22 04:35:41 wikidb: errorMessage:
2008-07-22 04:35:41 wikidb: headers:
2008-07-22 04:35:41 wikidb: timeout: 1
2008-07-22 04:35:41 wikidb: toFile:
2008-07-22 04:35:41 wikidb: wgTitle->getFullURL():
http://server/wiki/index.php/Special:Collection
2008-07-22 04:35:41 wikidb: curl_exec:
Resource id #92
2008-07-22 04:35:42 wikidb: url: http://server.domain.name/cgi-bin/mwlib.cgi
2008-07-22 04:35:42 wikidb: postFields:
2008-07-22 04:35:42 wikidb: postFields[metabook]:
{"type":"collection","licenses":
[{"type":"license","name":"License","mw_rights_icon":"","mw_rights_page":"","mw_rights_url":"","mw_rights_text":""}],"items":
[{"type":"article","content-type":"text\/x-wiki","title":"Company
Starting Notes","timestamp":1212005783}]}
2008-07-22 04:35:42 wikidb: postFields[base_url]: http://server/wiki
2008-07-22 04:35:42 wikidb: postFields[template_blacklist]:
MediaWiki:PDF Template Blacklist
2008-07-22 04:35:42 wikidb: postFields[writer]: rl
2008-07-22 04:35:42 wikidb: postFields[command]: render
2008-07-22 04:35:42 wikidb: postFields[login_credentials]: mwlib-
user:mwlib-password
2008-07-22 04:35:42 wikidb: errorMessage:
2008-07-22 04:35:42 wikidb: headers: Array
2008-07-22 04:35:42 wikidb: headers[date]: Tue, 22 Jul 2008
04:35:41 GMT
2008-07-22 04:35:42 wikidb: headers[server]: Apache/2.2.8 (Win32)
DAV/2 mod_ssl/2.2.8 OpenSSL/0.9.8g mod_autoindex_color PHP/5.2.5
2008-07-22 04:35:42 wikidb: headers[content-length]: 53
2008-07-22 04:35:42 wikidb: headers[content-type]: application/
json
2008-07-22 04:35:42 wikidb: timeout: 1
2008-07-22 04:35:42 wikidb: toFile:
2008-07-22 04:35:42 wikidb: text: {"collection_id":
"dd6455a9faa9f1d8", "writer": "rl"}
2008-07-22 04:35:42 wikidb: args:
2008-07-22 04:35:42 wikidb: $args[collection_id]: dd6455a9faa9f1d8
2008-07-22 04:35:42 wikidb: $args[writer]: rl
2008-07-22 04:35:42 wikidb: $args[command]: render_status
2008-07-22 04:35:42 wikidb: $args[login_credentials]: mwlib-
user:mwlib-password
2008-07-22 04:35:42 wikidb: url: http://server.domain.name/cgi-bin/mwlib.cgi
2008-07-22 04:35:42 wikidb: postFields:
2008-07-22 04:35:42 wikidb: postFields[collection_id]:
dd6455a9faa9f1d8
2008-07-22 04:35:42 wikidb: postFields[writer]: rl
2008-07-22 04:35:42 wikidb: postFields[command]: render_status
2008-07-22 04:35:42 wikidb: postFields[login_credentials]: mwlib-
user:mwlib-password
2008-07-22 04:35:42 wikidb: errorMessage:
2008-07-22 04:35:42 wikidb: headers:
2008-07-22 04:35:42 wikidb: timeout: 1
2008-07-22 04:35:42 wikidb: toFile:
2008-07-22 04:35:42 wikidb: wgTitle->getFullURL():
http://server/wiki/index.php/Special:Collection
2008-07-22 04:35:42 wikidb: curl_exec:
Resource id #88
2008-07-22 04:35:42 wikidb: url: http://server.domain.name/cgi-bin/mwlib.cgi
2008-07-22 04:35:42 wikidb: postFields:
2008-07-22 04:35:42 wikidb: postFields[collection_id]:
dd6455a9faa9f1d8
2008-07-22 04:35:42 wikidb: postFields[writer]: rl
2008-07-22 04:35:42 wikidb: postFields[command]: render_status
2008-07-22 04:35:42 wikidb: postFields[login_credentials]: mwlib-
user:mwlib-password
2008-07-22 04:35:42 wikidb: errorMessage:
2008-07-22 04:35:42 wikidb: headers: Array
2008-07-22 04:35:42 wikidb: headers[date]: Tue, 22 Jul 2008
04:35:42 GMT
2008-07-22 04:35:42 wikidb: headers[server]: Apache/2.2.8 (Win32)
DAV/2 mod_ssl/2.2.8 OpenSSL/0.9.8g mod_autoindex_color PHP/5.2.5
2008-07-22 04:35:42 wikidb: headers[content-length]: 101
2008-07-22 04:35:42 wikidb: headers[content-type]: application/
json
2008-07-22 04:35:42 wikidb: timeout: 1
2008-07-22 04:35:42 wikidb: toFile:
2008-07-22 04:35:42 wikidb: text: {"status": {"progress": 0},
"collection_id": "dd6455a9faa9f1d8", "state": "progress", "writer":
"rl"}
2008-07-22 04:35:45 wikidb: args:
2008-07-22 04:35:45 wikidb: $args[collection_id]: dd6455a9faa9f1d8
2008-07-22 04:35:45 wikidb: $args[writer]: rl
2008-07-22 04:35:45 wikidb: $args[command]: render_status
2008-07-22 04:35:45 wikidb: $args[login_credentials]: mwlib-
user:mwlib-password
2008-07-22 04:35:45 wikidb: url: http://server.domain.name/cgi-bin/mwlib.cgi
2008-07-22 04:35:45 wikidb: postFields:
2008-07-22 04:35:45 wikidb: postFields[collection_id]:
dd6455a9faa9f1d8
2008-07-22 04:35:45 wikidb: postFields[writer]: rl
2008-07-22 04:35:45 wikidb: postFields[command]: render_status
2008-07-22 04:35:45 wikidb: postFields[login_credentials]: mwlib-
user:mwlib-password
2008-07-22 04:35:45 wikidb: errorMessage:
2008-07-22 04:35:45 wikidb: headers:
2008-07-22 04:35:45 wikidb: timeout: 1
2008-07-22 04:35:45 wikidb: toFile:
2008-07-22 04:35:45 wikidb: wgTitle->getFullURL():
http://server/wiki/index.php/Special:Collection
2008-07-22 04:35:45 wikidb: curl_exec:
Resource id #87
2008-07-22 04:35:45 wikidb: url: http://server.domain.name/cgi-bin/mwlib.cgi
2008-07-22 04:35:45 wikidb: postFields:
2008-07-22 04:35:45 wikidb: postFields[collection_id]:
dd6455a9faa9f1d8
2008-07-22 04:35:45 wikidb: postFields[writer]: rl
2008-07-22 04:35:45 wikidb: postFields[command]: render_status
2008-07-22 04:35:45 wikidb: postFields[login_credentials]: mwlib-
user:mwlib-password
2008-07-22 04:35:45 wikidb: errorMessage:
2008-07-22 04:35:45 wikidb: headers: Array
2008-07-22 04:35:45 wikidb: headers[date]: Tue, 22 Jul 2008
04:35:45 GMT
2008-07-22 04:35:45 wikidb: headers[server]: Apache/2.2.8 (Win32)
DAV/2 mod_ssl/2.2.8 OpenSSL/0.9.8g mod_autoindex_color PHP/5.2.5
2008-07-22 04:35:45 wikidb: headers[content-length]: 101
2008-07-22 04:35:45 wikidb: headers[content-type]: application/
json
2008-07-22 04:35:45 wikidb: timeout: 1
2008-07-22 04:35:45 wikidb: toFile:
2008-07-22 04:35:45 wikidb: text: {"status": {"progress": 0},
"collection_id": "dd6455a9faa9f1d8", "state": "progress", "writer":
"rl"}
2008-07-22 04:35:48 wikidb: args:
2008-07-22 04:35:48 wikidb: $args[collection_id]: dd6455a9faa9f1d8
2008-07-22 04:35:48 wikidb: $args[writer]: rl
2008-07-22 04:35:48 wikidb: $args[command]: render_status
2008-07-22 04:35:48 wikidb: $args[login_credentials]: mwlib-
user:mwlib-password
2008-07-22 04:35:48 wikidb: url: http://server.domain.name/cgi-bin/mwlib.cgi
2008-07-22 04:35:48 wikidb: postFields:
2008-07-22 04:35:48 wikidb: postFields[collection_id]:
dd6455a9faa9f1d8
2008-07-22 04:35:48 wikidb: postFields[writer]: rl
2008-07-22 04:35:48 wikidb: postFields[command]: render_status
2008-07-22 04:35:48 wikidb: postFields[login_credentials]: mwlib-
user:mwlib-password
2008-07-22 04:35:48 wikidb: errorMessage:
2008-07-22 04:35:48 wikidb: headers:
2008-07-22 04:35:48 wikidb: timeout: 1
2008-07-22 04:35:48 wikidb: toFile:
2008-07-22 04:35:48 wikidb: wgTitle->getFullURL():
http://server/wiki/index.php/Special:Collection
2008-07-22 04:35:48 wikidb: curl_exec:
Resource id #88
2008-07-22 04:35:48 wikidb: url: http://server.domain.name/cgi-bin/mwlib.cgi
2008-07-22 04:35:48 wikidb: postFields:
2008-07-22 04:35:48 wikidb: postFields[collection_id]:
dd6455a9faa9f1d8
2008-07-22 04:35:48 wikidb: postFields[writer]: rl
2008-07-22 04:35:48 wikidb: postFields[command]: render_status
2008-07-22 04:35:48 wikidb: postFields[login_credentials]: mwlib-
user:mwlib-password
2008-07-22 04:35:48 wikidb: errorMessage:
2008-07-22 04:35:48 wikidb: headers: Array
2008-07-22 04:35:48 wikidb: headers[date]: Tue, 22 Jul 2008
04:35:48 GMT
2008-07-22 04:35:48 wikidb: headers[server]: Apache/2.2.8 (Win32)
DAV/2 mod_ssl/2.2.8 OpenSSL/0.9.8g mod_autoindex_color PHP/5.2.5
2008-07-22 04:35:48 wikidb: headers[content-length]: 101
2008-07-22 04:35:48 wikidb: headers[content-type]: application/
json
2008-07-22 04:35:48 wikidb: timeout: 1
2008-07-22 04:35:48 wikidb: toFile:
2008-07-22 04:35:48 wikidb: text: {"status": {"progress": 0},
"collection_id": "dd6455a9faa9f1d8", "state": "progress", "writer":
"rl"}
2008-07-22 04:35:52 wikidb: args:
2008-07-22 04:35:52 wikidb: $args[collection_id]: dd6455a9faa9f1d8
2008-07-22 04:35:52 wikidb: $args[writer]: rl
2008-07-22 04:35:52 wikidb: $args[command]: render_status
2008-07-22 04:35:52 wikidb: $args[login_credentials]: mwlib-
user:mwlib-password
2008-07-22 04:35:52 wikidb: url: http://server.domain.name/cgi-bin/mwlib.cgi
2008-07-22 04:35:52 wikidb: postFields:
2008-07-22 04:35:52 wikidb: postFields[collection_id]:
dd6455a9faa9f1d8
2008-07-22 04:35:52 wikidb: postFields[writer]: rl
2008-07-22 04:35:52 wikidb: postFields[command]: render_status
2008-07-22 04:35:52 wikidb: postFields[login_credentials]: mwlib-
user:mwlib-password
2008-07-22 04:35:52 wikidb: errorMessage:
2008-07-22 04:35:52 wikidb: headers:
2008-07-22 04:35:52 wikidb: timeout: 1
2008-07-22 04:35:52 wikidb: toFile:
2008-07-22 04:35:52 wikidb: wgTitle->getFullURL():
http://server/wiki/index.php/Special:Collection
2008-07-22 04:35:52 wikidb: curl_exec:
Resource id #88
2008-07-22 04:35:52 wikidb: url: http://server.domain.name/cgi-bin/mwlib.cgi
2008-07-22 04:35:52 wikidb: postFields:
2008-07-22 04:35:52 wikidb: postFields[collection_id]:
dd6455a9faa9f1d8
2008-07-22 04:35:53 wikidb: postFields[writer]: rl
2008-07-22 04:35:53 wikidb: postFields[command]: render_status
2008-07-22 04:35:53 wikidb: postFields[login_credentials]: mwlib-
user:mwlib-password
2008-07-22 04:35:53 wikidb: errorMessage:
2008-07-22 04:35:53 wikidb: headers: Array
2008-07-22 04:35:53 wikidb: headers[date]: Tue, 22 Jul 2008
04:35:52 GMT
2008-07-22 04:35:53 wikidb: headers[server]: Apache/2.2.8 (Win32)
DAV/2 mod_ssl/2.2.8 OpenSSL/0.9.8g mod_autoindex_color PHP/5.2.5
2008-07-22 04:35:53 wikidb: headers[content-length]: 101
2008-07-22 04:35:53 wikidb: headers[content-type]: application/
json
2008-07-22 04:35:53 wikidb: timeout: 1
2008-07-22 04:35:53 wikidb: toFile:
2008-07-22 04:35:53 wikidb: text: {"status": {"progress": 0},
"collection_id": "dd6455a9faa9f1d8", "state": "progress", "writer":
"rl"}


Mediawiki.log - the interesting part of this call is that the call to
api.php does not contain any login credentials, and does not seem to
use the action=login command to retrieve an authentication token/
cookie.
-------------------
Start request
GET /wiki/index.php/Special:Collection/render_article/?
arttitle=Company_Starting_Notes
Host: server
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:
1.8.1.15) Gecko/20080623 Firefox/2.0.0.15
Accept: text/xml,application/xml,application/xhtml+xml,text/
html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://server/wiki/index.php/Company_Starting_Notes
Cookie: wikEdAutoUpdate=Mon%2C%2021%20Jul%202008%2019%3A37%3A01%20GMT;
wikEdFullscreen=1; wikEdSummaryHistory=Undo%2520revision%2520188%2520by
%2520%255B%255BSpecial%253AContributions%252FBrion%257CBrion%255D%255D
%2520(%255B%255BUser%2520talk%253ABrion%257CTalk%255D%255D);
wikidbUserName=Brion; wikidb_session=5572d59936e4ca6414339a88353ccec0;
wikEdFindHistory=%25E2%2597%258A%25E2%2597%258A%252012097;
wikidbUserID=2

Main cache: FakeMemCachedClient
Message cache: MediaWikiBagOStuff
Parser cache: MediaWikiBagOStuff
Unstubbing $wgParser on call of $wgParser->setFunctionHook from
wfSetupPathFunctions
Unstubbing $wgContLang on call of $wgContLang->getMagic from
MagicWord::load
Unstubbing $wgMessageCache on call of $wgMessageCache->addMessages
from wfSetupPathFunctions
Fully initialised

Language::loadLocalisation(): got localisation for en from source

Unstubbing $wgUser on call of $wgUser->isAllowed from

Title::userCanRead
Cache miss for user 2
Unstubbing $wgLoadBalancer on call of $wgLoadBalancer->getConnection
from wfGetDB
Logged in from session
Unstubbing $wgLang on call of $wgLang->getCode from
MessageCache::loadMessagesFile
MessageCache::load(): got from global cache
Unstubbing $wgOut on call of $wgOut->redirect from
Collection::renderCollection
OutputPage::sendCacheControl: private caching; **

Profile section ended by close(): -total

20080722043542 0.462 /wiki/index.php/Special:Collection/
render_article/?arttitle=Company_Starting_Notes

Request ended normally


Start request
GET /wiki/index.php/Special:Collection/rendering/?return_to=Company
+Starting+Notes&collection_id=dd6455a9faa9f1d8&writer=rl
Host: server
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:
1.8.1.15) Gecko/20080623 Firefox/2.0.0.15
Accept: text/xml,application/xml,application/xhtml+xml,text/
html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://server/wiki/index.php/Company_Starting_Notes
Cookie: wikEdAutoUpdate=Mon%2C%2021%20Jul%202008%2019%3A37%3A01%20GMT;
wikEdFullscreen=1; wikEdSummaryHistory=Undo%2520revision%2520188%2520by
%2520%255B%255BSpecial%253AContributions%252FBrion%257CBrion%255D%255D
%2520(%255B%255BUser%2520talk%253ABrion%257CTalk%255D%255D);
wikidbUserName=Brion; wikidb_session=5572d59936e4ca6414339a88353ccec0;
wikEdFindHistory=%25E2%2597%258A%25E2%2597%258A%252012097;
wikidbUserID=2

Main cache: FakeMemCachedClient
Message cache: MediaWikiBagOStuff
Parser cache: MediaWikiBagOStuff
Unstubbing $wgParser on call of $wgParser->setFunctionHook from
wfSetupPathFunctions
Unstubbing $wgContLang on call of $wgContLang->getMagic from
MagicWord::load
Unstubbing $wgMessageCache on call of $wgMessageCache->addMessages
from wfSetupPathFunctions
Fully initialised

Language::loadLocalisation(): got localisation for en from source

Unstubbing $wgUser on call of $wgUser->isAllowed from

Title::userCanRead
Cache miss for user 2
Unstubbing $wgLoadBalancer on call of $wgLoadBalancer->getConnection
from wfGetDB
Logged in from session
Unstubbing $wgLang on call of $wgLang->getCode from
MessageCache::loadMessagesFile
MessageCache::load(): got from global cache
Unstubbing $wgOut on call of $wgOut->setArticleRelated from
SpecialPage::setHeaders
OutputPage::sendCacheControl: private caching; **

Profile section ended by close(): -total

20080722043542 0.495 /wiki/index.php/Special:Collection/rendering/?
return_to=Company Starting
Notes&collection_id=dd6455a9faa9f1d8&writer=rl

Request ended normally

Start request
GET /wiki/api.php?action=query&meta=siteinfo&format=json
Accept-Encoding: identity
Host: server
Connection: close
User-Agent: mwlib

Main cache: FakeMemCachedClient
Message cache: MediaWikiBagOStuff
Parser cache: MediaWikiBagOStuff
Unstubbing $wgParser on call of $wgParser->setFunctionHook from
wfSetupPathFunctions
Unstubbing $wgContLang on call of $wgContLang->getMagic from
MagicWord::load
Unstubbing $wgMessageCache on call of $wgMessageCache->addMessages
from wfSetupPathFunctions
Fully initialised
Unstubbing $wgUser on call of $wgUser->isAllowed from
ApiMain::__construct
Profile section ended by close(): API:json
Profile section ended by close(): API:main
Profile section ended by close(): api.php
Profile section ended by close(): -total
20080722043543 0.064 /wiki/api.php?
action=query&meta=siteinfo&format=json anon

Start request
GET /wiki/index.php/Special:Collection/rendering/?
collection_id=dd6455a9faa9f1d8&writer=rl&return_to=Company+Starting
+Notes
Host: server
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:
1.8.1.15) Gecko/20080623 Firefox/2.0.0.15
Accept: text/xml,application/xml,application/xhtml+xml,text/
html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: wikEdAutoUpdate=Mon%2C%2021%20Jul%202008%2019%3A37%3A01%20GMT;
wikEdFullscreen=1; wikEdSummaryHistory=Undo%2520revision%2520188%2520by
%2520%255B%255BSpecial%253AContributions%252FBrion%257CBrion%255D%255D
%2520(%255B%255BUser%2520talk%253ABrion%257CTalk%255D%255D);
wikidbUserName=Brion; wikidb_session=5572d59936e4ca6414339a88353ccec0;
wikEdFindHistory=%25E2%2597%258A%25E2%2597%258A%252012097;
wikidbUserID=2

Main cache: FakeMemCachedClient
Message cache: MediaWikiBagOStuff
Parser cache: MediaWikiBagOStuff
Unstubbing $wgParser on call of $wgParser->setFunctionHook from
wfSetupPathFunctions
Unstubbing $wgContLang on call of $wgContLang->getMagic from
MagicWord::load
Unstubbing $wgMessageCache on call of $wgMessageCache->addMessages
from wfSetupPathFunctions
Fully initialised

Language::loadLocalisation(): got localisation for en from source

Unstubbing $wgUser on call of $wgUser->isAllowed from

Title::userCanRead
Cache miss for user 2
Unstubbing $wgLoadBalancer on call of $wgLoadBalancer->getConnection
from wfGetDB
Logged in from session
Unstubbing $wgLang on call of $wgLang->getCode from
MessageCache::loadMessagesFile
MessageCache::load(): got from global cache
Unstubbing $wgOut on call of $wgOut->setArticleRelated from
SpecialPage::setHeaders
OutputPage::sendCacheControl: private caching; **

Profile section ended by close(): -total

20080722043545 0.471 /wiki/index.php/Special:Collection/rendering/?
collection_id=dd6455a9faa9f1d8&writer=rl&return_to=Company Starting
Notes

Request ended normally


Start request
GET /wiki/index.php/Special:Collection/rendering/?
collection_id=dd6455a9faa9f1d8&writer=rl&return_to=Company+Starting
+Notes
Host: server
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:
1.8.1.15) Gecko/20080623 Firefox/2.0.0.15
Accept: text/xml,application/xml,application/xhtml+xml,text/
html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: wikEdAutoUpdate=Mon%2C%2021%20Jul%202008%2019%3A37%3A01%20GMT;
wikEdFullscreen=1; wikEdSummaryHistory=Undo%2520revision%2520188%2520by
%2520%255B%255BSpecial%253AContributions%252FBrion%257CBrion%255D%255D
%2520(%255B%255BUser%2520talk%253ABrion%257CTalk%255D%255D);
wikidbUserName=Brion; wikidb_session=5572d59936e4ca6414339a88353ccec0;
wikEdFindHistory=%25E2%2597%258A%25E2%2597%258A%252012097;
wikidbUserID=2
Cache-Control: max-age=0

Main cache: FakeMemCachedClient
Message cache: MediaWikiBagOStuff
Parser cache: MediaWikiBagOStuff
Unstubbing $wgParser on call of $wgParser->setFunctionHook from
wfSetupPathFunctions
Unstubbing $wgContLang on call of $wgContLang->getMagic from
MagicWord::load
Unstubbing $wgMessageCache on call of $wgMessageCache->addMessages
from wfSetupPathFunctions
Fully initialised

Language::loadLocalisation(): got localisation for en from source

Unstubbing $wgUser on call of $wgUser->isAllowed from

Title::userCanRead
Cache miss for user 2
Unstubbing $wgLoadBalancer on call of $wgLoadBalancer->getConnection
from wfGetDB
Logged in from session
Unstubbing $wgLang on call of $wgLang->getCode from
MessageCache::loadMessagesFile
MessageCache::load(): got from global cache
Unstubbing $wgOut on call of $wgOut->setArticleRelated from
SpecialPage::setHeaders
OutputPage::sendCacheControl: private caching; **

Profile section ended by close(): -total

20080722043548 0.491 /wiki/index.php/Special:Collection/rendering/?
collection_id=dd6455a9faa9f1d8&writer=rl&return_to=Company Starting
Notes

Request ended normally


Start request
GET /wiki/index.php/Special:Collection/rendering/?
collection_id=dd6455a9faa9f1d8&writer=rl&return_to=Company+Starting
+Notes
Host: server
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:
1.8.1.15) Gecko/20080623 Firefox/2.0.0.15
Accept: text/xml,application/xml,application/xhtml+xml,text/
html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: wikEdAutoUpdate=Mon%2C%2021%20Jul%202008%2019%3A37%3A01%20GMT;
wikEdFullscreen=1; wikEdSummaryHistory=Undo%2520revision%2520188%2520by
%2520%255B%255BSpecial%253AContributions%252FBrion%257CBrion%255D%255D
%2520(%255B%255BUser%2520talk%253ABrion%257CTalk%255D%255D);
wikidbUserName=Brion; wikidb_session=5572d59936e4ca6414339a88353ccec0;
wikEdFindHistory=%25E2%2597%258A%25E2%2597%258A%252012097;
wikidbUserID=2
Cache-Control: max-age=0

Main cache: FakeMemCachedClient
Message cache: MediaWikiBagOStuff
Parser cache: MediaWikiBagOStuff
Unstubbing $wgParser on call of $wgParser->setFunctionHook from
wfSetupPathFunctions
Unstubbing $wgContLang on call of $wgContLang->getMagic from
MagicWord::load
Unstubbing $wgMessageCache on call of $wgMessageCache->addMessages
from wfSetupPathFunctions
Fully initialised

Language::loadLocalisation(): got localisation for en from source

Unstubbing $wgUser on call of $wgUser->isAllowed from

Title::userCanRead
Cache miss for user 2
Unstubbing $wgLoadBalancer on call of $wgLoadBalancer->getConnection
from wfGetDB
Logged in from session
Unstubbing $wgLang on call of $wgLang->getCode from
MessageCache::loadMessagesFile
MessageCache::load(): got from global cache
Unstubbing $wgOut on call of $wgOut->setArticleRelated from
SpecialPage::setHeaders
OutputPage::sendCacheControl: private caching; **

Profile section ended by close(): -total

20080722043553 0.471 /wiki/index.php/Special:Collection/rendering/?
collection_id=dd6455a9faa9f1d8&writer=rl&return_to=Company Starting
Notes

Request ended normally

On Jul 21, 7:13 am, Johannes Beigel <johannes.bei...@brainbot.com>
wrote:

> Am 21.07.2008 um 08:35 schrieb brion.fin...@gmail.com:
>
> > * modifying recorddb.py from mwlib for cross-platform issue:
> > ** line 333-ish: os.rename(output + '.tmp', output)
> > ** changeto:
> > os.unlink(output)
> > os.rename(output + '.tmp', output)
>
> I added this (with an if os.path.exists()) in the hg repository.
>
> > ** $wgCollectionMWServeCredentials = "mwlib-user:mwlib-password";
> > * modifying wiki setup to allow unauthenticated users to view web
> > (because I am still having authentication issues)
>
> Could you tell us a little bit more about the problems? Log output etc?
>
> > * modifying the following files for cross-platform issues:
> > ** wsgi.py: ~ line 50 there is a content length mismatch. The byte
> > count difference implies that it might be due to line termination
> > characters.
>

> Could be the same problem we had inhttp://code.pediapress.com/wiki/ticket/191

[Brion]

But -- it does not resolve the broken PDF file problem. I read ticket
191 which you referenced above, which sounds like a similar problem,
but this did not resolve it.

The version of Collection that I am using is 37841, which should
include the fixes described in 191.

[Johannes Beigel]

Am 22.07.2008 um 07:21 schrieb Brion:
> But -- it does not resolve the broken PDF file problem. I read ticket
> 191 which you referenced above, which sounds like a similar problem,
> but this did not resolve it.

Yes, this seems to be still a problem. I've created a new ticket,
because the old one (191) talked too much about (non-existent) Python
bugs :-)

http://code.pediapress.com/wiki/ticket/230

[Johannes Beigel]

Am 22.07.2008 um 07:12 schrieb Brion:
> A little bit more about the authentication problem.

The output "invalid base URL" looks suspicious.

I've added code to the login stuff in mwlib that checks the result of
the login call. Could you checkout the latest version and execute the
following simple test program (after replacing the placeholder strings)?

#! /usr/bin/env python
from mwlib.mwapidb import APIHelper
a = APIHelper('BASEURL')
print 'Usable:', a.is_usable()
print 'Login:', a.login('USERNAME', 'PASSWORD')

The output should contain True in the last two print statements.

If it prints False after the a.login() call, could you replace the
last line with this code:

post_data = urllib.urlencode({
'action': 'login',
'lgname': 'USERNAME,
'lgpassword': 'PASSWORD',
'format': 'json',
})
result = a.opener.open('%sapi.php' % a.base_url, post_data)
print result.read()

and post the output?

Regards,
Johannes Beigel

[Johannes Beigel]

Am 22.07.2008 um 07:03 schrieb Brion:
> mw-render.log - this log file claims that the base URL is invalid, but
> it only claims this when read access is restricted. When read access
> is not restricted, it does not complain.

Ah ok, I overlooked this one. I don't understand it though. Maybe the
output of the test code I posted in my other mail will give some clues.

[Johannes Beigel]

If you're using the CGI script, this change in mwlib.cgi could
possibly fix the problem:

http://code.pediapress.com/hg/mwlib/rev/8ffcbb609e1a

[Brion]

I installed mwlib changeset 1303 ac55b037cc90 by extracting the
archive and running make; python setup.py install

I am still using 0.8.0 mwlib.rl

I ran the test programs below. (The windows cmd shell "neatly"
formats everything into 80 columns, which is more relevant for the
second program)

Program 1:
----------------
SERVER# python
Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit
(Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from mwlib.mwapidb import APIHelper
>>> a=APIHelper('http://server/wiki')
>>> print 'Usable:', a.is_usable()
Usable:mwlib.utils.info >> fetching 'http://server/wiki/api.php?
action=query&met
a=siteinfo&format=json'
mwlib.utils.info >> got 'http://server/wiki/api.php?
action=query&meta=siteinfo&f
ormat=json' (57 Bytes)
mwapidb.error >> Got exception: ValueError('No JSON object could be
decoded',)
False
>>> print 'Login:', a.login('mwlib-user','mwlib-password')
Login:

Traceback (most recent call last):

File "<stdin>", line 1, in <module>
File "mwlib\mwapidb.py", line 168, in login
result = simplejson.loads(result.read())
File "C:\Python25\lib\site-packages\PIL\__init__.py", line 311, in
loads

File "build\bdist.win32\egg\simplejson\decoder.py", line 321, in
decode
File "build\bdist.win32\egg\simplejson\decoder.py", line 340, in
raw_decode
ValueError: No JSON object could be decoded

Program 2: - interesting things I see - there is a surprising error
about LDAP TLS. I didn't think my LDAP config was using TLS. But the
mwlib user isn't actually in the LDAP, the mwlib-user is stored in the
wiki db as a local user. I have "dual" authentication, if you will.
I have tested the mwlib-user login through the wiki, and it works
fine.

In support of that, it looks like the last result is a successful
login token for the mwlib-user. Also, I have the corresponding
mediawiki library below, which shows the api.php query and api.php
login.
----------------
SERVER# python
Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit
(Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from mwlib.mwapidb import APIHelper
>>> a=APIHelper('http://server/wiki')
>>> print 'Usable:', a.is_usable()
Usable:mwlib.utils.info >> fetching 'http://server/wiki/api.php?
action=query&met
a=siteinfo&format=json'
mwlib.utils.info >> got 'http://server/wiki/api.php?
action=query&meta=siteinfo&f
ormat=json' (57 Bytes)
mwapidb.error >> Got exception: ValueError('No JSON object could be
decoded',)
False
>>> post_data = urllib.urlencode({
... 'action': 'login',
... 'lgname': 'mwlib-user',
... 'lgpassword': 'mwlib-password',
... 'format': 'json',
... })

Traceback (most recent call last):

File "<stdin>", line 1, in <module>
NameError: name 'urllib' is not defined
>>> import urllib
>>> post_data = urllib.urlencode({
... 'action': 'login',
... 'lgname': 'mwlib-user',
... 'lgpassword': 'mwlib-password',
... 'format': 'json',
... })

>>> result = a.opener.open('%sapi.php' % a.base_url, post_data)
>>> print result.read()

<br />
<font size='1'><table class='xdebug-error' dir='ltr' border='1'
cellspacing='0'
cellpadding='1'>
<tr><th align='left' bgcolor='#f57900' colspan="5"><span
style='background-color
: #cc0000; color: #fce94f; font-size: x-large;'>( ! )</span> Warning:
ldap_start
_tls() [<a href='function.ldap-start-tls'>function.ldap-start-tls</
a>]: Unable t
o start TLS: Can't contact LDAP server in C:\xampp\htdocs\wiki
\extensions\LdapAu
thentication\LdapAuthentication.php on line <i>197</i></th></tr>
<tr><th align='left' bgcolor='#e9b96e' colspan='5'>Call Stack</th></
tr>
<tr><th align='center' bgcolor='#eeeeec'>#</th><th align='left'
bgcolor='#eeeeec
'>Time</th><th align='left' bgcolor='#eeeeec'>Memory</th><th
align='left' bgcolo
r='#eeeeec'>Function</th><th align='left' bgcolor='#eeeeec'>Location</
th></tr>
<tr><td bgcolor='#eeeeec' align='center'>1</td><td bgcolor='#eeeeec'
align='cent
er'>0.0002</td><td bgcolor='#eeeeec' align='right'>62992</td><td
bgcolor='#eeeee
c'>{main}( )</td><td title='C:\xampp\htdocs\wiki\api.php'
bgcolor='#eeeeec'>..\
api.php<b>:</b>0</td></tr>
<tr><td bgcolor='#eeeeec' align='center'>2</td><td bgcolor='#eeeeec'
align='cent
er'>0.0995</td><td bgcolor='#eeeeec' align='right'>7368976</td><td
bgcolor='#eee
eec'>ApiMain->execute( )</td><td title='C:\xampp\htdocs\wiki\api.php'
bgcolor='
#eeeeec'>..\api.php<b>:</b>77</td></tr>
<tr><td bgcolor='#eeeeec' align='center'>3</td><td bgcolor='#eeeeec'
align='cent
er'>0.0996</td><td bgcolor='#eeeeec' align='right'>7369544</td><td
bgcolor='#eee
eec'>ApiMain->executeActionWithErrorHandling( )</td><td title='C:
\xampp\htdocs\
wiki\includes\api\ApiMain.php' bgcolor='#eeeeec'>..\ApiMain.php<b>:</
b>208</td><
/tr>
<tr><td bgcolor='#eeeeec' align='center'>4</td><td bgcolor='#eeeeec'
align='cent
er'>0.0996</td><td bgcolor='#eeeeec' align='right'>7411848</td><td
bgcolor='#eee
eec'>ApiMain->executeAction( )</td><td title='C:\xampp\htdocs\wiki
\includes\api
\ApiMain.php' bgcolor='#eeeeec'>..\ApiMain.php<b>:</b>223</td></tr>
<tr><td bgcolor='#eeeeec' align='center'>5</td><td bgcolor='#eeeeec'
align='cent
er'>0.1036</td><td bgcolor='#eeeeec' align='right'>7761152</td><td
bgcolor='#eee
eec'>ApiLogin->execute( )</td><td title='C:\xampp\htdocs\wiki\includes
\api\ApiM
ain.php' bgcolor='#eeeeec'>..\ApiMain.php<b>:</b>355</td></tr>
<tr><td bgcolor='#eeeeec' align='center'>6</td><td bgcolor='#eeeeec'
align='cent
er'>0.1455</td><td bgcolor='#eeeeec' align='right'>11313152</td><td
bgcolor='#ee
eeec'>LoginForm->authenticateUserData( )</td><td title='C:\xampp
\htdocs\wiki\in
cludes\api\ApiLogin.php' bgcolor='#eeeeec'>..\ApiLogin.php<b>:</b>100</
td></tr>
<tr><td bgcolor='#eeeeec' align='center'>7</td><td bgcolor='#eeeeec'
align='cent
er'>0.1511</td><td bgcolor='#eeeeec' align='right'>11330784</td><td
bgcolor='#ee
eeec'>User->checkPassword( )</td><td title='C:\xampp\htdocs\wiki
\includes\Speci
alUserlogin.php' bgcolor='#eeeeec'>..\SpecialUserlogin.php<b>:</b>401</
td></tr>
<tr><td bgcolor='#eeeeec' align='center'>8</td><td bgcolor='#eeeeec'
align='cent
er'>0.1511</td><td bgcolor='#eeeeec' align='right'>11331512</td><td
bgcolor='#ee
eeec'>LdapAuthenticationPlugin->authenticate( )</td><td title='C:
\xampp\htdocs\
wiki\includes\User.php' bgcolor='#eeeeec'>..\User.php<b>:</b>2266</
td></tr>
<tr><td bgcolor='#eeeeec' align='center'>9</td><td bgcolor='#eeeeec'
align='cent
er'>0.1511</td><td bgcolor='#eeeeec' align='right'>11341416</td><td
bgcolor='#ee
eeec'>LdapAuthenticationPlugin->connect( )</td><td title='C:\xampp
\htdocs\wiki\
extensions\LdapAuthentication\LdapAuthentication.php'
bgcolor='#eeeeec'>..\LdapA
uthentication.php<b>:</b>259</td></tr>
<tr><td bgcolor='#eeeeec' align='center'>10</td><td bgcolor='#eeeeec'
align='cen
ter'>0.1512</td><td bgcolor='#eeeeec' align='right'>11343744</td><td
bgcolor='#e
eeeec'><a href='http://www.php.net/ldap_start_tls'
target='_new'>ldap_start_tls<
/a>
( )</td><td title='C:\xampp\htdocs\wiki\extensions\LdapAuthentication
\LdapAuthe
ntication.php' bgcolor='#eeeeec'>..\LdapAuthentication.php<b>:</b>197</
td></tr>
</table></font>
{"login":{"result":"Success","lguserid":12,"lgusername":"mwlib-
user","lgtoken":"96dac
17c39e1cf2ef07d42af468a8091","cookieprefix":"wikidb","sessionid":"471b5492cd78cd
4f7049aa6ca960f816"}}

Mediawiki log

-------------------
Start request
GET /wiki/api.php?action=query&meta=siteinfo&format=json
Accept-Encoding: identity
Host: server
Connection: close
User-Agent: mwlib

Main cache: FakeMemCachedClient
Message cache: MediaWikiBagOStuff
Parser cache: MediaWikiBagOStuff
Unstubbing $wgParser on call of $wgParser->setFunctionHook from
wfSetupPathFunctions
Unstubbing $wgContLang on call of $wgContLang->getMagic from
MagicWord::load
Unstubbing $wgMessageCache on call of $wgMessageCache->addMessages
from wfSetupPathFunctions
Fully initialised
Unstubbing $wgUser on call of $wgUser->isAllowed from
ApiMain::__construct
Profile section ended by close(): API:json
Profile section ended by close(): API:main
Profile section ended by close(): api.php
Profile section ended by close(): -total

20080722175345 0.086 /wiki/api.php?
action=query&meta=siteinfo&format=json anon


Start request
POST /wiki/api.php
Accept-Encoding: identity
Content-Length: 61
Host: server
Content-Type: application/x-www-form-urlencoded

Connection: close
User-Agent: mwlib

Main cache: FakeMemCachedClient
Message cache: MediaWikiBagOStuff
Parser cache: MediaWikiBagOStuff
Unstubbing $wgParser on call of $wgParser->setFunctionHook from
wfSetupPathFunctions
Unstubbing $wgContLang on call of $wgContLang->getMagic from
MagicWord::load
Unstubbing $wgMessageCache on call of $wgMessageCache->addMessages
from wfSetupPathFunctions
Fully initialised
Unstubbing $wgUser on call of $wgUser->isAllowed from
ApiMain::__construct

IP: 192.168.0.66
Unstubbing $wgLang on call of $wgLang->specialPage from
LoginForm::LoginForm

Language::loadLocalisation(): got localisation for en from source

Unstubbing $wgLoadBalancer on call of $wgLoadBalancer->getConnection
from wfGetDB

MessageCache::load(): got from global cache

Cache miss for user 12

Profile section ended by close(): -total

20080722175345 0.102 /wiki/api.php

Also, I tested the latest mwlib install on the mediawiki (tried to
export to pdf) and got this error:
--------------------
File "c:\python25\lib\site-packages\mwlib-0.8.1.dev-py2.5-win32.egg
\mwlib\apps.py", line 350, in render
env = parser.makewiki()
File "c:\python25\lib\site-packages\mwlib-0.8.1.dev-py2.5-win32.egg
\mwlib\options.py", line 117, in makewiki
password=password,
File "c:\python25\lib\site-packages\mwlib-0.8.1.dev-py2.5-win32.egg

\mwlib\wiki.py", line 245, in makewiki

res = _makewiki(conf, metabook, username=username,
password=password)
File "c:\python25\lib\site-packages\mwlib-0.8.1.dev-py2.5-win32.egg
\mwlib\wiki.py", line 201, in _makewiki
res.wiki = wiki_mwapi(url, username=username, password=password)
File "c:\python25\lib\site-packages\mwlib-0.8.1.dev-py2.5-win32.egg
\mwlib\wiki.py", line 26, in wiki_mwapi
password=password,
File "c:\python25\lib\site-packages\mwlib-0.8.1.dev-py2.5-win32.egg
\mwlib\mwapidb.py", line 429, in __init__
assert self.login(username, password), 'login failed'
File "c:\python25\lib\site-packages\mwlib-0.8.1.dev-py2.5-win32.egg
\mwlib\mwapidb.py", line 448, in login
return self.api_helper.login(username, password)
File "c:\python25\lib\site-packages\mwlib-0.8.1.dev-py2.5-win32.egg
\mwlib\mwapidb.py", line 168, in login
result = simplejson.loads(result.read())
File "build\bdist.win32\egg\simplejson\__init__.py", line 311, in
loads
File "build\bdist.win32\egg\simplejson\decoder.py", line 321, in
decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "build\bdist.win32\egg\simplejson\decoder.py", line 340, in
raw_decode
raise ValueError("No JSON object could be decoded")



On Jul 22, 6:50 am, Johannes Beigel <johannes.bei...@brainbot.com>
wrote:

[Johannes Beigel]

Am 22.07.2008 um 20:07 schrieb Brion:
>>>> from mwlib.mwapidb import APIHelper
>>>> a=APIHelper('http://server/wiki')
>>>> print 'Usable:', a.is_usable()
> Usable:mwlib.utils.info >> fetching 'http://server/wiki/api.php?
> action=query&met
> a=siteinfo&format=json'
> mwlib.utils.info >> got 'http://server/wiki/api.php?
> action=query&meta=siteinfo&f
> ormat=json' (57 Bytes)
> mwapidb.error >> Got exception: ValueError('No JSON object could be
> decoded',)
> False

This is because you have to be logged in for meta=siteinfo to work.
mwapidb did it wrong until yesterday, too, which has been fixed now.

> I have "dual" authentication, if you will.

Which is most probably not supported.

>>>> import urllib
>>>> post_data = urllib.urlencode({
> ... 'action': 'login',
> ... 'lgname': 'mwlib-user',
> ... 'lgpassword': 'mwlib-password',
> ... 'format': 'json',
> ... })
>>>> result = a.opener.open('%sapi.php' % a.base_url, post_data)
>>>> print result.read()

This result should return data in JSON format. If you get something
else (your response looked like HTML containing an error description
for some LdapAuthentication extension) you have to debug your
MediaWiki setup. mwlib is only able to talk plain HTTP with api.php
and expects JSON as result.

[Brion]

Perfect!

I debugged the LDAPAuthentication extension. The reason it was
returning a TLS error is because the domain was not set for the login
request, and the default for the LDAP Authentication when encryption
is not specified is TLS, and then after that its fallback mode must be
to authenticate against the database.

If I add lgdomain to the login post request, that resolves the error
message. I tested the request using this HTML form:

<html>
<form action="http://server/wiki/api.php" method=post><br>
Action: <input type="text" name="action" value="login"><br>
Login Name: <input type="text" name="lgname" value="mwlib-user"><br>
Login Password: <input type="text" name="lgpassword" value="mwlib-
password"><br>
Login Domain: <input type="text" name="lgdomain" value="mwlib-
domain"><br>
Return Format: <input type="text" name="format" value="json"><br>
<INPUT type="submit" value="Send"> <INPUT type="reset">
</form>
</html>

and that returned the Json result without the error message:

{"login":{"result":"Success","lguserid":12,"lgusername":"mwlib-

user","lgtoken":"96dac17c39e1cf3ef07d42bf467a8091","cookieprefix":"wikidb","sessionid":"6e5f3dd6c28ee325ba141e3d4c099848"}}

When the domain is not specified, the LDAPAuthentication extension
uses the key "invaliddomain", so as a workaround I configured the
appropriate encryption type for the domain "invaliddomain". That also
resolves the error message so that only the JSON is returned for the
login request.

Could lgdomain be added to the user credentials? Instead of
configuring an "invaliddomain" in the LDAP extension, I could specify
the lgdomain in the $wgCollectionMWServeCredentials, like
$wgCollectionMWServeCredentials=user:password:domain. This would get
passed to api.php as lgdomain=domain.


Final result:
With the "invaliddomain" workaround in place, mwlib properly
authenticates and generates a PDF file on Windows on a mediawiki setup
that uses the LDAPAuthentication extension using

mwlib changeset 1303 (ac55b037cc90 )
mwlib.rl 0.8.0
Collection revision 37841

Certain templates are not rendering, but I saw some documentation
about that that I will go read.

I will review my notes and post up a how-to configure for windows.

Thanks for the help,

Brion

On Jul 23, 8:00 am, Johannes Beigel <johannes.bei...@brainbot.com>
wrote:

[Johannes Beigel]

Am 23.07.2008 um 17:17 schrieb Brion:
> Could lgdomain be added to the user credentials?

Sounds reasonable. I created a ticket:

http://code.pediapress.com/wiki/ticket/235

> I will review my notes and post up a how-to configure for windows.

That would be cool!

Regards,
Johannes

[Johannes Beigel]

Brion, could you check if this problem has been fixed for you? If it
has, the ticket

http://code.pediapress.com/wiki/ticket/228

can be closed as fixed.

Regards,
Johannes

[Johannes Beigel]

Am 23.07.2008 um 18:52 schrieb Johannes Beigel:
> Am 23.07.2008 um 17:17 schrieb Brion:
>> Could lgdomain be added to the user credentials?
>
> Sounds reasonable. I created a ticket:
>
> http://code.pediapress.com/wiki/ticket/235

Has been implemented in

http://code.pediapress.com/hg/mwlib/rev/b94c4679eacd

Could you test, if it works as intended?

Regards,
Johannes

[Brion]

I posted a first cut at the instructions here:
http://code.pediapress.com/wiki/wiki/WindowsSetup

and I updated the troubleshooting note for "Python was built with
Visual Studio 2003" to point to this page.
Here is a url to the troubleshooting note: http://code.pediapress.com/wiki/wiki/Troubleshooting

I did not add a link from the mwlib setup page, but I could.

On Jul 23, 11:52 am, Johannes Beigel <johannes.bei...@brainbot.com>
wrote:

[Brion]

Yes, #228 is fixed in http://code.pediapress.com/hg/mwlib/rev/ac55b037cc90

When I wrote the mwlib-on-windows instructions, I verified the
instructions by installing on a new windows vista machine. I
installed the changeset referenced above and verified that it
generates PDFs. It does not fail on the content-length check
described in #228.

On Jul 24, 9:02 am, Johannes Beigel <johannes.bei...@brainbot.com>
wrote:

[Brion]

I installed mwlib changeset 1315:b94c4679eacd by extracting the
archive and running make; python setup.py install.

test 1: verify that setup with "invalid domain" workaround,
user:password credentials (no domain) still works with new changeset -
verified

test 2: verify that setup with "invalid domain" workaround, no
security credentials fails
(this verifies that the security credentials were still needed with
the "invalid domain" workaround)
-verified
-fails (as expected) with message that no JSON object could be decoded

test 3: verify that setup with user:password credentials (no domain)
fails after removing "invalid domain" workaround
(this verifies that the "invalid domain" workaround was still needed
with the incomplete security credentials (ie, no domain))
-verified
-fails (as expected) with message that no JSON object could be decoded

test 4: verify that setup with user:password:domain credentials
passes. (The "invalid domain" workaround has been removed.)
(this verifies that the changeset with the added domain support in the
security credentials works without an "invalid domain" workaround)
-passes

summary:
confirmed - the fix in changeset 1315 works as expected.

Very cool! Thanks.

Brion

On Jul 24, 9:26 am, Johannes Beigel <johannes.bei...@brainbot.com>
wrote:

[Johannes Beigel]

Am 27.07.2008 um 06:28 schrieb Brion:
> I posted a first cut at the instructions here:
> http://code.pediapress.com/wiki/wiki/WindowsSetup

Cool, thank you very much, Brion!

One small point: Maybe the installation of mwlib could be separated
from the installation of MediaWiki/the Collection extension. There
might be lots of people who want to use mwlib with an existing
MediaWiki setup or people who want to use the Collection with the
existing mw-serve on tools.pediparess.com.

Regards,
Johannes

[Johannes Beigel]

Am 27.07.2008 um 07:00 schrieb Brion:
> summary:
> confirmed - the fix in changeset 1315 works as expected.

Thanks for confirming this!

Regards,
Johannes

[Brion]

Good suggestion. I updated the page and broke it into four sections:
1. installing the development/build/runtime environment on windows
2. installing mwlib
3. installing mwlib.rl
4. installing the collection extension.
I added links to the pages for each of the components in each section.

I also figured out how to add a table of contents and I added a link
back to the google group for questions and further information. I
also invited people to update the wiki page if they had corrections or
information to add.

Brion

On Jul 27, 4:09 am, Johannes Beigel <johannes.bei...@brainbot.com>
wrote:

[Johannes Beigel]

Am 27.07.2008 um 20:54 schrieb Brion:
> Good suggestion. I updated the page and broke it into four sections:
> 1. installing the development/build/runtime environment on windows
> 2. installing mwlib
> 3. installing mwlib.rl
> 4. installing the collection extension.
> I added links to the pages for each of the components in each section.
>
> I also figured out how to add a table of contents and I added a link
> back to the google group for questions and further information. I
> also invited people to update the wiki page if they had corrections or
> information to add.

Wow, good work! Thanks a lot!

Johannes

[Ralf Schmitt]

First, thanks for all your work on this...

On Sun, Jul 27, 2008 at 8:54 PM, Brion <brion....@gmail.com> wrote:
>
> Good suggestion. I updated the page and broke it into four sections:
> 1. installing the development/build/runtime environment on windows

Would it help if we provided egg files for mwlib/mwlib.rl for windows?
Brion, would you volunteer to test them? I think the whole msys/mingw
install could be skipped this way. I only have to boot my vmware image
and type python setup.py bdist_egg to build them. However, I don't
feel like testing it on windows.

Regards,
- Ralf

[Brion]

Sure, if you put together the egg files I could test them out.

I agree it would make the install process easier.

On Jul 28, 3:41 am, "Ralf Schmitt" <sch...@gmail.com> wrote:
> First, thanks for all your work on this...
>

[Monic Pia]

I know that this is very very old topic, but I want to know: Is there any instruction of installing mwlib on Windows? Because http://code.pediapress.com/wiki/wiki/WindowsSetup doesn't work now.