[TracTicketChangeLogPlugin] eats up server
14 views
Skip to first unread message
Torge Riedel
Jan 20, 2015, 4:26:34 PM1/20/15
to trac-...@googlegroups.com
Hi @all,
just got a server shutdown by my provider caused by 100% CPU load by apache processes.
After reactivation I played around and found the following:
- I have TracTicketChangeLogPlugin installed (the latest version from repo, updated on 18th January 2015)
- If I view a ticket with 3 entries in the commit history I get an apache process with 100% CPU load, staying at this level
- Killing this process raises another one reaching 100% CPU load, staying at this level
- Killing the second process too reduces the server load to a normal level
- After deactivation of the plugin I can view any ticket without a problem
-> seems to be related to the ticket
Any ideas?
I'm sure that I had no problems with this plugin of an older version.
Regards
Torge
just got a server shutdown by my provider caused by 100% CPU load by apache processes.
After reactivation I played around and found the following:
- I have TracTicketChangeLogPlugin installed (the latest version from repo, updated on 18th January 2015)
- If I view a ticket with 3 entries in the commit history I get an apache process with 100% CPU load, staying at this level
- Killing this process raises another one reaching 100% CPU load, staying at this level
- Killing the second process too reduces the server load to a normal level
- After deactivation of the plugin I can view any ticket without a problem
-> seems to be related to the ticket
Any ideas?
I'm sure that I had no problems with this plugin of an older version.
Regards
Torge
Ryan Ollos
Jan 20, 2015, 11:26:35 PM1/20/15
to trac-...@googlegroups.com
Which branch have you installed from, 0.12 or 1.0? What is the version of the plugin? I don't see anything in recent changes that looks concerning, but you could try rolling back to that earlier version of the plugin and check if the issue occurs with that one ticket. What version / revision were you running prior to updating the plugin on 18th of Jan? Can you reproduce the issue with more that one of the tickets in your system?
The TracTicketChangeLogPlugin runs a LIKE query on the revision table on every request. I'm no database performance expert, but that has concerned me since I first started looking at the plugin. An outline of changes for improving the plugin can be found here: http://trac-hacks.org/ticket/11821
Torge Riedel
Jan 21, 2015, 1:40:03 AM1/21/15
to trac-...@googlegroups.com
Am 21.01.2015 um 05:26 schrieb Ryan Ollos:
> --
> You received this message because you are subscribed to the Google Groups "Trac Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+...@googlegroups.com <mailto:trac-users+...@googlegroups.com>.
> To post to this group, send email to trac-...@googlegroups.com <mailto:trac-...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/trac-users.
> For more options, visit https://groups.google.com/d/optout.
Installed vom Branch 1.0 ("TracTicketChangelogPlugin 1.0.0dev" is displayed in the "Plugins" section of the "Admin" menu).
I will try stepping back revision by revision to see when this "issue was added."
But I think at first it is a good idea to update from trac 1.0.1 to 1.0.3 to make sure there are no side effects.
Thanks for your reply
Torge
> You received this message because you are subscribed to the Google Groups "Trac Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+...@googlegroups.com <mailto:trac-users+...@googlegroups.com>.
> To post to this group, send email to trac-...@googlegroups.com <mailto:trac-...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/trac-users.
> For more options, visit https://groups.google.com/d/optout.
Installed vom Branch 1.0 ("TracTicketChangelogPlugin 1.0.0dev" is displayed in the "Plugins" section of the "Admin" menu).
I will try stepping back revision by revision to see when this "issue was added."
But I think at first it is a good idea to update from trac 1.0.1 to 1.0.3 to make sure there are no side effects.
Thanks for your reply
Torge
Ryan Ollos
Jan 21, 2015, 1:54:04 AM1/21/15
to trac-...@googlegroups.com
On Tue, Jan 20, 2015 at 10:39 PM, Torge Riedel <torge...@gmx.de> wrote:
Am 21.01.2015 um 05:26 schrieb Ryan Ollos:
--On Tue, Jan 20, 2015 at 12:09 PM, Torge Riedel <torge...@gmx.de <mailto:torge...@gmx.de>> wrote:
Hi @all,
just got a server shutdown by my provider caused by 100% CPU load by apache processes.
After reactivation I played around and found the following:
- I have TracTicketChangeLogPlugin installed (the latest version from repo, updated on 18th January 2015)
- If I view a ticket with 3 entries in the commit history I get an apache process with 100% CPU load, staying at this level
- Killing this process raises another one reaching 100% CPU load, staying at this level
- Killing the second process too reduces the server load to a normal level
- After deactivation of the plugin I can view any ticket without a problem
-> seems to be related to the ticket
Any ideas?
I'm sure that I had no problems with this plugin of an older version.
Regards
Torge
Which branch have you installed from, 0.12 or 1.0? What is the version of the plugin? I don't see anything in recent changes that looks concerning, but you could try rolling back to that earlier version of the plugin and check if the issue occurs with that one ticket. What version / revision were you running prior to updating the plugin on 18th of Jan? Can you reproduce the issue with more that one of the tickets in your system?
The TracTicketChangeLogPlugin runs a LIKE query on the revision table on every request. I'm no database performance expert, but that has concerned me since I first started looking at the plugin. An outline of changes for improving the plugin can be found here: http://trac-hacks.org/ticket/11821
You received this message because you are subscribed to the Google Groups "Trac Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscribe@googlegroups.com <mailto:trac-users+unsub...@googlegroups.com>.
To post to this group, send email to trac-...@googlegroups.com <mailto:trac-users@googlegroups.com>.
Visit this group at http://groups.google.com/group/trac-users.
For more options, visit https://groups.google.com/d/optout.
Installed vom Branch 1.0 ("TracTicketChangelogPlugin 1.0.0dev" is displayed in the "Plugins" section of the "Admin" menu).
I will try stepping back revision by revision to see when this "issue was added."
Thanks, that will be very helpful info. In the event that you started on
the 0.12 branch and later switched to the 1.0 branch, it would be
useful to test the 0.12 branch again. In the 0.12 branch the features
are mostly implemented in JavaScript. In the 1.0 branch the plugin is
mostly implemented in Python and a Genshi template.
Torge Riedel
Feb 15, 2015, 8:19:27 AM2/15/15
to trac-...@googlegroups.com
Am 21.01.2015 um 07:53 schrieb Ryan Ollos:
> To post to this group, send email to trac-...@googlegroups.com <mailto:trac-...@googlegroups.com> <mailto:trac-users@__googlegroups.com <mailto:trac-...@googlegroups.com>>.
> Visit this group at http://groups.google.com/__group/trac-users <http://groups.google.com/group/trac-users>.
> For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.
> To post to this group, send email to trac-...@googlegroups.com <mailto:trac-...@googlegroups.com>.
Today I found the time to check it again. At first I updated to Trac-1.0.4 and activated TracTicketChangeLogPlugin. I get the 100% CPU usage again as soon as I want to view a ticket with related commits. I tried with 0.12 branch of the plugin and it works without any problems.
Here is my log pattern (list all commits where this ticket is mentioned):
[ticketlog]
log_pattern = (\n|.)*#%s($|\D(\n|.)*)
Trac about page:
Paket Version
Trac 1.0.4
Babel 0.9.6
FullBlog 0.1.1
Genshi 0.6 (with speedups)
mod_wsgi 3.3 (WSGIProcessGroup WSGIApplicationGroup %{GLOBAL})
MySQL server: "5.5.41-0ubuntu0.12.04.1", client: "5.5.41", thread-safe: 1
MySQLdb 1.2.3
Pygments 1.4
Python 2.7.3 (default, Dec 18 2014, 19:25:50) [GCC 4.6.3]
setuptools 0.6
Subversion 1.6.17 (r1128011)
System:
Ubuntu 12.04 LTS Server
Any ideas? Let me know if you need more information.
Regards
Torge
> On Tue, Jan 20, 2015 at 10:39 PM, Torge Riedel <torge...@gmx.de <mailto:torge...@gmx.de>> wrote:
>
> Am 21.01.2015 um 05:26 schrieb Ryan Ollos:
>
>
> Am 21.01.2015 um 05:26 schrieb Ryan Ollos:
>
> On Tue, Jan 20, 2015 at 12:09 PM, Torge Riedel <torge...@gmx.de <mailto:torge...@gmx.de> <mailto:torge...@gmx.de <mailto:torge...@gmx.de>>> wrote:
>
> Hi @all,
>
> just got a server shutdown by my provider caused by 100% CPU load by apache processes.
>
> After reactivation I played around and found the following:
>
> - I have TracTicketChangeLogPlugin installed (the latest version from repo, updated on 18th January 2015)
> - If I view a ticket with 3 entries in the commit history I get an apache process with 100% CPU load, staying at this level
> - Killing this process raises another one reaching 100% CPU load, staying at this level
> - Killing the second process too reduces the server load to a normal level
> - After deactivation of the plugin I can view any ticket without a problem
> -> seems to be related to the ticket
>
> Any ideas?
> I'm sure that I had no problems with this plugin of an older version.
>
> Regards
> Torge
>
>
> Which branch have you installed from, 0.12 or 1.0? What is the version of the plugin? I don't see anything in recent changes that looks concerning, but you could try rolling back to that earlier version of the plugin and check if the issue occurs with that one ticket. What version / revision were you running prior to updating the plugin on 18th of Jan? Can you reproduce the issue with more that one of the tickets in your system?
>
> The TracTicketChangeLogPlugin runs a LIKE query on the revision table on every request. I'm no database performance expert, but that has concerned me since I first started looking at the plugin. An outline of changes for improving the plugin can be found here: http://trac-hacks.org/ticket/__11821 <http://trac-hacks.org/ticket/11821>
>
> Hi @all,
>
> just got a server shutdown by my provider caused by 100% CPU load by apache processes.
>
> After reactivation I played around and found the following:
>
> - I have TracTicketChangeLogPlugin installed (the latest version from repo, updated on 18th January 2015)
> - If I view a ticket with 3 entries in the commit history I get an apache process with 100% CPU load, staying at this level
> - Killing this process raises another one reaching 100% CPU load, staying at this level
> - Killing the second process too reduces the server load to a normal level
> - After deactivation of the plugin I can view any ticket without a problem
> -> seems to be related to the ticket
>
> Any ideas?
> I'm sure that I had no problems with this plugin of an older version.
>
> Regards
> Torge
>
>
> Which branch have you installed from, 0.12 or 1.0? What is the version of the plugin? I don't see anything in recent changes that looks concerning, but you could try rolling back to that earlier version of the plugin and check if the issue occurs with that one ticket. What version / revision were you running prior to updating the plugin on 18th of Jan? Can you reproduce the issue with more that one of the tickets in your system?
>
>
> --
> You received this message because you are subscribed to the Google Groups "Trac Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscribe@__googlegroups.com <mailto:trac-users%2Bunsu...@googlegroups.com> <mailto:trac-users+_...@googlegroups.com <mailto:trac-users%2Bunsu...@googlegroups.com>>.
> --
> You received this message because you are subscribed to the Google Groups "Trac Users" group.
> To post to this group, send email to trac-...@googlegroups.com <mailto:trac-...@googlegroups.com> <mailto:trac-users@__googlegroups.com <mailto:trac-...@googlegroups.com>>.
> Visit this group at http://groups.google.com/__group/trac-users <http://groups.google.com/group/trac-users>.
> For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.
>
>
> Installed vom Branch 1.0 ("TracTicketChangelogPlugin 1.0.0dev" is displayed in the "Plugins" section of the "Admin" menu).
>
> I will try stepping back revision by revision to see when this "issue was added."
>
>
> Thanks, that will be very helpful info. In the event that you started on the 0.12 branch and later switched to the 1.0 branch, it would be useful to test the 0.12 branch again. In the 0.12 branch the features are mostly implemented in JavaScript. In the 1.0 branch the plugin is mostly implemented in Python and a Genshi template.
>
> But I think at first it is a good idea to update from trac 1.0.1 to 1.0.3 to make sure there are no side effects.
>
>
>
>
> Installed vom Branch 1.0 ("TracTicketChangelogPlugin 1.0.0dev" is displayed in the "Plugins" section of the "Admin" menu).
>
> I will try stepping back revision by revision to see when this "issue was added."
>
>
> Thanks, that will be very helpful info. In the event that you started on the 0.12 branch and later switched to the 1.0 branch, it would be useful to test the 0.12 branch again. In the 0.12 branch the features are mostly implemented in JavaScript. In the 1.0 branch the plugin is mostly implemented in Python and a Genshi template.
>
> But I think at first it is a good idea to update from trac 1.0.1 to 1.0.3 to make sure there are no side effects.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "Trac Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+...@googlegroups.com <mailto:trac-users+...@googlegroups.com>.
> You received this message because you are subscribed to the Google Groups "Trac Users" group.
> To post to this group, send email to trac-...@googlegroups.com <mailto:trac-...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/trac-users.
> For more options, visit https://groups.google.com/d/optout.
Sorry, was very busy for a while.
> For more options, visit https://groups.google.com/d/optout.
Today I found the time to check it again. At first I updated to Trac-1.0.4 and activated TracTicketChangeLogPlugin. I get the 100% CPU usage again as soon as I want to view a ticket with related commits. I tried with 0.12 branch of the plugin and it works without any problems.
Here is my log pattern (list all commits where this ticket is mentioned):
[ticketlog]
log_pattern = (\n|.)*#%s($|\D(\n|.)*)
Trac about page:
Paket Version
Trac 1.0.4
Babel 0.9.6
FullBlog 0.1.1
Genshi 0.6 (with speedups)
mod_wsgi 3.3 (WSGIProcessGroup WSGIApplicationGroup %{GLOBAL})
MySQL server: "5.5.41-0ubuntu0.12.04.1", client: "5.5.41", thread-safe: 1
MySQLdb 1.2.3
Pygments 1.4
Python 2.7.3 (default, Dec 18 2014, 19:25:50) [GCC 4.6.3]
setuptools 0.6
Subversion 1.6.17 (r1128011)
System:
Ubuntu 12.04 LTS Server
Any ideas? Let me know if you need more information.
Regards
Torge
RjOllos
Mar 3, 2015, 11:49:40 AM3/3/15
to trac-...@googlegroups.com, torge...@gmx.de
On Sunday, February 15, 2015 at 5:19:27 AM UTC-8, Torge Riedel wrote:
Am 21.01.2015 um 07:53 schrieb Ryan Ollos:
> On Tue, Jan 20, 2015 at 10:39 PM, Torge Riedel <torge...@gmx.de <mailto:torge...@gmx.de>> wrote:
>
> Am 21.01.2015 um 05:26 schrieb Ryan Ollos:
>
> On Tue, Jan 20, 2015 at 12:09 PM, Torge Riedel <torge...@gmx.de <mailto:torge...@gmx.de> <mailto:torge...@gmx.de <mailto:torge...@gmx.de>>> wrote:
>
> Hi @all,
>
> just got a server shutdown by my provider caused by 100% CPU load by apache processes.
>
> After reactivation I played around and found the following:
>
> - I have TracTicketChangeLogPlugin installed (the latest version from repo, updated on 18th January 2015)
> - If I view a ticket with 3 entries in the commit history I get an apache process with 100% CPU load, staying at this level
> - Killing this process raises another one reaching 100% CPU load, staying at this level
> - Killing the second process too reduces the server load to a normal level
> - After deactivation of the plugin I can view any ticket without a problem
> -> seems to be related to the ticket
>
> Any ideas?
> I'm sure that I had no problems with this plugin of an older version.
>
> Regards
> Torge
>
>
> Which branch have you installed from, 0.12 or 1.0? What is the version of the plugin? I don't see anything in recent changes that looks concerning, but you could try rolling back to that earlier version of the plugin and check if the issue occurs with that one ticket. What version / revision were you running prior to updating the plugin on 18th of Jan? Can you reproduce the issue with more that one of the tickets in your system?
>
> The TracTicketChangeLogPlugin runs a LIKE query on the revision table on every request. I'm no database performance expert, but that has concerned me since I first started looking at the plugin. An outline of changes for improving the plugin can be found here: http://trac-hacks.org/ticket/__11821 <http://trac-hacks.org/ticket/11821>
>
> --
> You received this message because you are subscribed to the Google Groups "Trac Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscribe@__googlegroups.com <mailto:trac-users%2Bunsu...@googlegroups.com> <mailto:trac-users+__unsub...@googlegroups.com <mailto:trac-users%2Bunsu...@googlegroups.com>>.
> To post to this group, send email to trac-...@googlegroups.com <mailto:trac-...@googlegroups.com> <mailto:trac-...@__googlegroups.com <mailto:trac-...@googlegroups.com>>.
> Visit this group at http://groups.google.com/__group/trac-users <http://groups.google.com/group/trac-users>.
> For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.
>
>
> Installed vom Branch 1.0 ("TracTicketChangelogPlugin 1.0.0dev" is displayed in the "Plugins" section of the "Admin" menu).
>
> I will try stepping back revision by revision to see when this "issue was added."
>
>
> Thanks, that will be very helpful info. In the event that you started on the 0.12 branch and later switched to the 1.0 branch, it would be useful to test the 0.12 branch again. In the 0.12 branch the features are mostly implemented in JavaScript. In the 1.0 branch the plugin is mostly implemented in Python and a Genshi template.
>
> But I think at first it is a good idea to update from trac 1.0.1 to 1.0.3 to make sure there are no side effects.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "Trac Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+...@googlegroups.com <mailto:trac-users+unsub...@googlegroups.com>.
The major change on the 1.0 branch occurred in [14186], when the JavaScript implementation was replaced with a Genshi template prepared in an ITemplateStreamFilter.
You could try revisions 14185 and 14186 to determine whether that is when the problem was introduced (seems most likely). How many revisions exist in the repository that you are displaying ChangeLogs from?
RjOllos
Apr 13, 2015, 7:51:41 AM4/13/15
to trac-...@googlegroups.com, torge...@gmx.de
You can follow #11821 if you'd like to track resolution of this performance issue. In particular, see http://trac-hacks.org/ticket/11821#comment:21
Reply all
Reply to author
Forward
0 new messages