Sad state of the doc wiki
Christopher Arndt
In the past weeks the state of documentation wiki has rapidly detoriated
due to heavy spamming to the degree of not beeing useable/maintainable
anymore.
The amount of spamming is also beginning to cause strange bugs in
MoinMoin, like the trouble with the "ThirdParty" page (which, btw, has
still not been fixed almost HALF A YEAR since the problem is known) and
now the RecentChanges page seems to be broken as well [1].
I suspect that most of the problems are caused by the comment plugin,
which makes it just too easy for spammers and obviously has some serious
bugs too. I think, it is time to admit that the current comment system
was a bad idea and, in the absence of a current better solution, we
should simply disable and remove it. Many of the comments weren't that
useful anyway and there aren't that many.
Furthermore we need to actively maintain the wiki in a better way.
Accounts should be regularly monitored to check if any spammers have
obtained an account. Edits should only be possible by logged in users.
We don't have that many documentation contributors anyway, so I,
personally, don't care if some people, who wouldn't bother to get an
account, are discouraged from contributing.
To implement this measures, we (i.e. the members of the documentation
team) need to obtain the wiki administration priviledges from those who
currently hold [2] it and, to be frank, haven't been able/willing to do
anything about the problems that have been known for far too long already.
Sorry, if this message comes accross a bit harsh, but I'm just worried
that all the effort that already went into the documentation is just
going to waste by neglect.
Chris
[1] At least for me it now only ever shows "ValueError invalid literal
for long() [...]" as do several other pages
[2] I can't check who those are, because the "DocTeam" page is broken
too :-(
Lee McFadden
harsh... to be fair, sugar coating the situation can sometimes be a
bad thing :)
If everyone else agrees with Chris I will take a look at the moinmoin
instance to try and remove the comment system.
Lee
--
Lee McFadden
blog: http://www.splee.co.uk
work: http://fireflisystems.com
skype: fireflisystems
Christopher Arndt
> If everyone else agrees with Chris I will take a look at the moinmoin
> instance to try and remove the comment system.
I hope the */PageCommentData pages will not be removed as well? We could
harvest them automatically later and add their content to the parent
pages. Could probably done by a little script with BeautifulSoup and
mechanize.
BTW, I just hacked together a little script that downloads every page in
the wiki and checks, if it is broken (i.e. contains a DIV with
class="traceback"). This could be easiliy extended to do more checks and
we could run it on a regular basis. It should cache the downloaded pages
though. The script is attached.
It currently coughs up the following list of broken pages:
/1.0/AlternativeTemplating
/1.0/CLIReference
/1.0/Configuration
/1.0/GenerateFigures
/1.0/GettingStarted/Admin
/1.0/GettingStarted/Configuration
/1.0/TgAdmin
/1.0/ThirdParty
/DocTeam
/VideoHelp
Chris
Felix Schwarz
I think the spamming problem is serious so requiring everybody to login is okay
with me. If the comment system causes much of the trouble, feel free to remove it.
Although, there are some pages with really helpful comments. We should save these.
Furthermore, I would like to see a capability to have a 'discussion' mechanism
similar to the current comment system (or MediaWiki's discussion page). But
clearly, for me this priority is lower than having a spam-free wiki.
fs
Florent Aide
> !!!This is a call for action concerning the documentation wiki!!!
[...]
agreed. I regularly remove spams from comments but this is when I read
the page... and it come quicker than we can remove them anyway.
Let's remove the comments for the moment and add a protected comment
system with an anti-spam filter (distorted images that the user must
type the content of) when we find one...
Cheers,
Florent.
Christopher Arndt
> agreed. I regularly remove spams from comments but this is when I read
> the page... and it come quicker than we can remove them anyway.
>
> Let's remove the comments for the moment and add a protected comment
> system with an anti-spam filter (distorted images that the user must
> type the content of) when we find one...
Until the comment plugin has been disabled, I am now replacing the
comment macro on every page where I find spam with the following note:
.. note:: The comment feature has been disabled on this page due to
heavy spamming. If you want to comment on the contents of this page, if
you have questions, or want to report an error, please write to the
TurboGears `mailing list`_.
.. _mailing list: 1.0/GettingHelp
If there are already some comments, it would be good to integrate them
into the page or just paste them to the end of the page, but gettig rid
of spam has priority for me now.
Chris
Florent Aide
> Until the comment plugin has been disabled, I am now replacing the
> comment macro on every page where I find spam with the following note:
Could someone with access to the machine send us (in this list) the data
files for the pages that are chocking. We could get the content back.
I did this a lot when I had to manage a MoinMoin wiki it works well.
I even edited data files in place with vim sometimes but hush! :)
Cheers,
Florent
Florent Aide
> files for the pages that are chocking. We could get the content back.
Sorry I answer to myself here: If no-one can send us the data files,
then google appears to still have the pages in cache :)
/flo
Christopher Arndt
> Could someone with access to the machine send us (in this list) the data
> files for the pages that are chocking. We could get the content back.
You mean pages like ThirdParty and the list sent on Saturday?
How do you want to get the content back in when all you get now when you
access the page is a traceback?
Even http://docs.turbogears.org/1.0/ThirdParty?action=recall&rev=X
doesn't work.
Chris
Florent Aide
still there anyway...
Christopher Arndt
> Are you sure nothing remains in the data file ? Maybe something is
> still there anyway...
That's not what I meant. The data may still be there, but I see no
possibility to get it back onto the wiki page through the web interface,
because all you ever get when you try to access the page is the rotten
traceback. Maybe you could craft a POST request with the necessary form
fields to update the page yourself, but it's probably easier to just
delete the page on the server and then create it again.
Chris
Florent Aide
Hi Chris,
That is the reason I asked if someone on this list has access to the
machine (I mean SSH or physical) to get the data file to us so we can at
least read it an not start the doc from scratch :(
/flo
Christopher Arndt
> BTW, I just hacked together a little script that downloads every page in
> the wiki and checks, if it is broken
> It currently coughs up the following list of broken pages:
>
> /1.0/AlternativeTemplating
> /1.0/CLIReference
> /1.0/Configuration
> /1.0/GenerateFigures
> /1.0/GettingStarted/Admin
> /1.0/GettingStarted/Configuration
> /1.0/TgAdmin
> /1.0/ThirdParty
> /DocTeam
> /VideoHelp
Is anybody of the turbogears.org admins taking care of these? Who can I
bug to get these fixed?????
Chris
Christopher Arndt
> Christopher Arndt schrieb:
>> BTW, I just hacked together a little script that downloads every page in
>> the wiki and checks, if it is broken
>
>> It currently coughs up the following list of broken pages:
>>
>>
> bug to get these fixed?????
Just FYI, Elvelind has kindly fixed the pages. Many thanks to him!
If any of you has the time to check if the information on these pages is
still up-to-date, that would be very much apprecciated.
Chris
Christopher Arndt
- We seem to have got on top of the spammers for the moment. I removed
the comment macro on all pages that were heavily spammed and the comment
spam has gone down from dozens of spam messages per day to a few per week.
- As I said in another mail, Evelind has fixed the broken pages like
1.0/ThirdParty
- Comments can still be deleted by everybody. I asked Elvelind to have a
look into this as well.
- I haven't noticed any further vandalizing of wiki pages.
To sum it up: apart from the problem with deleting comments, the
situation seems to under control for the moment but we need to watch it
further. I think for the moment we can leave the comment plug-in
enabled, since there are still a few useful comments trickling in every
now and then. If anybody knows of a better alternative, please let us know.
Cheers, Chris
Elvelind Grandin
http://docs.turbogears.org/WikiSandBox <- you can delete all comments.
http://docs.turbogears.org/1.0/ThirdParty <- need a password to delete them.
not sure how it is with the other pages.
--
cheers
elvelind grandin
Christopher Arndt
> regarding the deleting of comments. it seems to depend on the page.
> http://docs.turbogears.org/WikiSandBox <- you can delete all comments.
> http://docs.turbogears.org/1.0/ThirdParty <- need a password to delete them.
This is probably because the latter has
#acl All:read EditorGroup:read,write
at the top of the page source, i.e. it is a protected page, which kind
of makes sense, if you think about it. BUT, even with unprotected
comments, non-editors can only delete comments and not edit them (by
editing the PagCommentData page), so it this "feature" is of limited use.
Chris
Christopher Arndt
> Just a little update on this topic:
>
> - We seem to have got on top of the spammers for the moment. I removed
> the comment macro on all pages that were heavily spammed and the comment
> spam has gone down from dozens of spam messages per day to a few per week.
Ok, the spammers caught up again :-( Today they flodded a few dozen
pages with spam comments. Removing the PageComment from all of them (and
saving the old, valid comments on the page) would take far too long and
would only help for a while again anyway.
Can somebody PLEASE disable that stupid PageComment plugin and restart
the wiki?
My sysadmin access is not working yet, so I can't do it.
Chris
Christopher Arndt
> Quick question about pagecomment macro.
>
> Do you guys allow anonymous write access to the pages?
Yes, everybody can edit pages that are not marked "Official". I
protected some pages that had been vandalized by spammers so that they
can only be edited by logged in userss. You can see the access rights
for each page in the page source right at the top. If there is nothing,
then everybody can edit. It's a wiki after all.
> Do you guys remember if the spam in comments was coming from anonymous
> spam users or registered spam users ?
It doesn't matter. Comments can be added to each page with the macro,
regardless of whether the page itself is write protected or not. That's
the main weakness of the comment macro. I usually don't add it anymore
to new pages, but I've been too lazy to remove it completely yet :-(
Chris
Lukasz Szybalski
On May 21 2007, 7:56 am, Christopher Arndt <chris.ar...@web.de> wrote:
> Florent Aide schrieb:
>
> > agreed. I regularly remove spams from comments but this is when I read
> > the page... and it come quicker than we can remove them anyway.
>
> > Let's remove the comments for the moment and add a protected comment
> > system with an anti-spam filter (distorted images that the user must
> > type the content of) when we find one...
>
> Until the comment plugin has been disabled, I am now replacing the
> comment macro on every page where I find spam with the following note:
>
> .. note:: The comment feature has been disabled on this page due to
> heavy spamming. If you want to comment on the contents of this page, if
> .. _mailing list: 1.0/GettingHelp
>
> If there are already some comments, it would be good to integrate them
> into the page or just paste them to the end of the page, but gettig rid
> of spam has priority for me now.
Do you guys allow anonymous write access to the pages?
spam users or registered spam users ?
Lukasz Szybalski
On 'my' wiki a user is required to login. This caused spam to go from
50 pages/spam edits to 0 a year ago. Now the logged in spam user is
spamming about 5 new pages per week. I modified the pagecomment macro
so that a user is required to login and was wondering if it would work
for turbogears wiki?
That is why I was wondering if you pagecomment spam was coming from
registered spammers or anonymous spammers.
Lucas
Christopher Arndt
> On Wed, Apr 30, 2008 at 9:13 AM, Christopher Arndt <chris.arndt-S0/GAf8...@public.gmane.org> wrote:
> On 'my' wiki a user is required to login. This caused spam to go from
> 50 pages/spam edits to 0 a year ago.
The idea was to let every contribute to the documentation as easily as
possible. Maybe that was a bit naive ;-) We get spammers once in a while
that completely overwrite unprotected pages, but not to often. Anyway,
it's easy to revert the page and then write-protect it if necessary. A
wiki needs some maintenance, if you're not prepared to do this, then a
wiki isnsn't the right thing, IMHO.
> Now the logged in spam user is
> spamming about 5 new pages per week.
We had those a few (<20) times. I disabled their user accounts and so
far it hasn't gotten out of control.
> I modified the pagecomment macro
> so that a user is required to login and was wondering if it would work
> for turbogears wiki?
Send code plz ;-)
Chris
Lukasz Szybalski
http://moinmo.in/MacroMarket/PageComment2
PageComment2-0.981-only_logged_in_user.py
I didn't do nothing fancy. I just commented out the code that filled
in the anonymous ip/password. Without these 2 items it doesn't look
like you can post a comment unless you're logged in.
Which leaves you with spammers that have user names already.
Lucas
Lukasz Szybalski
Were you able to try it out, and see if it would make a difference?
Lucas
Christopher Arndt
>> > > I modified the pagecomment macro
>> > > so that a user is required to login and was wondering if it would work
>> > > for turbogears wiki?
>> >
>> > Send code plz ;-)
>>
>> http://moinmo.in/MacroMarket/PageComment2
>>
>> PageComment2-0.981-only_logged_in_user.py
>>
>> I didn't do nothing fancy. I just commented out the code that filled
>> in the anonymous ip/password. Without these 2 items it doesn't look
>> like you can post a comment unless you're logged in.
>>
>> Which leaves you with spammers that have user names already.
>>
>> Lucas
>>
>
> Were you able to try it out, and see if it would make a difference?
Sorry, I didn't have time yet. But to be honest, I am not sure whether I
want to invest any time into this plug-in, since I want to get rid of it
alltogether anyway.
If we require users to be logged in to post a comment, we can also send
them to trac to open a ticket straight away. The effort to leave a
comment seems the same to me and having a ticket makes it easier for teh
documentation maintainers to follow up on the comment. And if the user
has an email address registered in trac, it is possible to contact him
to clarify things.
Chris
Lukasz Szybalski
>
> Lukasz Szybalski schrieb:
>>> > > I modified the pagecomment macro
>>> > > so that a user is required to login and was wondering if it would work
>>> > > for turbogears wiki?
>>> >
>>> > Send code plz ;-)
>>>
>>> http://moinmo.in/MacroMarket/PageComment2
>>>
>>> PageComment2-0.981-only_logged_in_user.py
>>>
>>> I didn't do nothing fancy. I just commented out the code that filled
>>> in the anonymous ip/password. Without these 2 items it doesn't look
>>> like you can post a comment unless you're logged in.
>>>
>>> Which leaves you with spammers that have user names already.
>>>
>>> Lucas
>>>
>>
>> Were you able to try it out, and see if it would make a difference?
>
> Sorry, I didn't have time yet. But to be honest, I am not sure whether I
> want to invest any time into this plug-in, since I want to get rid of it
> alltogether anyway.
>
You mentioned you want to get rid of page comment macro. Before you do
that could you replace the macro with the one I have posted which
would require a user to log in.(just overwrite the macro) That would
be appreciated. I would like to see if that solves our spam problem.
If not then we can disable it completally i guess. At least until
there is a better anti spam measure.
Lucas
> If we require users to be logged in to post a comment, we can also send
> them to trac to open a ticket straight away. The effort to leave a
> comment seems the same to me and having a ticket makes it easier for teh
> documentation maintainers to follow up on the comment. And if the user
> has an email address registered in trac, it is possible to contact him
> to clarify things.
>
> Chris
>
> >
>
--
Automotive Recall Database. Cars, Trucks, etc.
http://www.lucasmanual.com/recall/
TurboGears Manual-Howto
http://lucasmanual.com/pdf/TurboGears-Manual-Howto.pdf