[Security] Hidden comment excerpts potentially publicly visible
Gareth Rees
We’ve recently handled an issue where hidden comments were potentially visible to public users. We recommend checking that you don’t have content that’s been affected.
Summary and impact
==================
In 2014 a feature was added to Alaveteli to allow admins to hide multiple comments at the same time via checkboxes in the admin interface. While this code correctly changed the visibility state of the selected comments, it failed to initiate a reindexing of the comments.
This meant that excerpts of comment content may have been displayed in some places where we use the indexed events to determine what to display.
There are three places where this happens:
1. Lists of request (e.g. /list/successful, /body/foo, etc)
2. User profile pages
3. Search results
In all cases only an excerpt of the content would have been visible.
In the case of lists of requests and user profile pages, only the first 150 characters where the comment is the most recent publicly renderable event on the request would have been shown. For search results it would be 150 characters centred around a search result match.
In many cases other events happening to the request would have caused a reindex for that request, or would have become the latest publicly renderable event and so displaced the comment as the content to render in the excerpt.
Lists of requests and search results are also limited to 20 pages, so over time it’s increasingly likely that older comments would get pushed out of the immediately accessible pages.
Fix
===
We fixed the issue in 2021, but this only resolved the issue for comments hidden via the checkbox mechanism going forward. Comments already hidden by the checkbox method may still have been present in the search index.
We’ve also implemented the ability for admins to destroy comments via the admin interface, which will be included in a future release.
Comments can always be destroyed via the Rails console if necessary. (e.g. `Comment.find(1).destroy`)
Checking affected content
-------------------------
You can run the following script to check whether any hidden comments you have are currently included in the search engine index.
Copy the following to a file that can be run by the application system user (e.g. `~/check_hidden_comments.rb`)
ActsAsXapian.readable_init
comments = Comment.where(visible: false)
events = InfoRequestEvent.joins(:comment).merge(comments)
indexed_events = events.select do |e|
ActsAsXapian.enquire.query = Xapian::Query.new("I" + e.xapian_document_term)
ActsAsXapian.enquire.mset(0,1,1).matches.any?
end
puts "Count of hidden comments in search index:"
puts indexed_events.size
puts "IDs of comments in search index:"
puts indexed_events.map(&:comment_id)
Then run it with the rails runner command as the application user:
bin/rails runner ~/check_hidden_comments.rb
Reindexing affected content
---------------------------
If you have hidden comments that are in the search index where they shouldn’t be, you can force a reindex of those comments with the following script.
Copy the following to a file that can be run by the application system user (e.g. `~/reindex_hidden_comments.rb`)
ActsAsXapian.readable_init
comments = Comment.where(visible: false)
events = InfoRequestEvent.joins(:comment).merge(comments)
indexed_events = events.select do |e|
ActsAsXapian.enquire.query = Xapian::Query.new("I" + e.xapian_document_term)
ActsAsXapian.enquire.mset(0,1,1).matches.any?
end
indexed_events.each(&:xapian_mark_needs_index)
Then run it with the rails runner command as the application user:
bin/rails runner ~/reindex_hidden_comments.rb
This will queue each event related to the comment for reindexing. By default reindexing happens every 5 minutes, so this may take some time to work its way through the queue.
We recommend running the check again after reindexing, as we did observe a handful of reindexing failures when running this on WhatDoTheyKnow.
Affected versions
=================
The bug was introduced in 0.18 and existed up to 0.40.1.3.
Comments hidden between these versions may continue to be affected, even if you're currently running a version beyond where the fix was introduced.
Fixed versions
==============
The bug was fixed in 0.41.0.0.
Since the bug has been fixed for some time, we will not be issuing a hotfix release for older versions. All sites should upgrade to 0.41.0.0 or later.