Why am I seeing a lot of repeated log lines in trac.log?

Velu Narasimman

unread,

Nov 4, 2019, 8:04:02 AM11/4/19

to Trac Users

Hi,

I am seeing a lot of same lines repeated in huge amounts in trac.log files. In general this repetition happens in all /<environment>/log/trac.log files.

I have attached the screenshot for your reference. How shall I suppress these excessive log lines? I welcome your idea.

Thank you!

repeated_log_lines.png

Ryan Ollos

unread,

Nov 4, 2019, 2:47:48 PM11/4/19

to Trac Users

Which Trac version?

What plugins do you have installed?

What is your [trac] permission_policies from trac.ini?

- Ryan

Velu Narasimman

unread,

Nov 5, 2019, 12:40:37 AM11/5/19

to Ryan Ollos, Trac Users

Hi Ryan,

Thanks for your response. I am using Trac 1.0.15. Below are the list of plugins that I have installed.

1. agilo-0.9.15

2. Babel-1.3

3. CherryPy-3.2.2

4. Genshi-0.7

5. psycopg2-2.7.4

6. pysugarcrm-1.0.0

7. TracMenusPlugin-0.1

Other than those plugins I have used some downloaded and modified plugins as well.

The permission policies from trac.ini are,

permission_policies = DefaultPermissionPolicy, LegacyAttachmentPolicy

The lines logged in that attachment are just a sample. I am seeing similar repetitions for a lot of other issues as well. They are repeated for ten to fifteen times instead of just one.

Also, my trac site is very slow. I am using Apache + mod_wsgi + Django. What may be the reason for this slowness? Could you please direct me to fix the performance?

Help me!

Velu Narasimman | Engineer, Information Systems

Mobile: +91-44-6740 400 | Phone: +91-44-6740 4000 | Extn. 11535 |

Website: www.aspiresys.com | Blog: http://blog.aspiresys.com

India’s Best Companies to Work for 2019!

From: trac-...@googlegroups.com <trac-...@googlegroups.com> on behalf of Ryan Ollos <rjo...@gmail.com>
Sent: Tuesday, November 5, 2019 1:17 AM
To: Trac Users <trac-...@googlegroups.com>
Subject: Re: [Trac] Why am I seeing a lot of repeated log lines in trac.log?

CAUTION: External email. Please verify the veracity of the sender/content before opening a link or an attachment.

--
You received this message because you are subscribed to the Google Groups "Trac Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trac-users/CA%2BBGpn808OKupbt2_3cJx-6Wyb5HarHY3fOic3db_GkyUK6kow%40mail.gmail.com.

This e-mail message and any attachments are for the sole use of the intended recipient(s) and may contain proprietary, confidential, trade secret or privileged information. Any unauthorized review, use, disclosure or distribution is prohibited and may be a violation of law. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.

RjOllos

unread,

Nov 5, 2019, 6:20:29 PM11/5/19

to Trac Users

On Monday, November 4, 2019 at 9:40:37 PM UTC-8, Velu Narasimman wrote:

Hi Ryan,

    Thanks for your response. I am using Trac 1.0.15. Below are the list of plugins that I have installed.

    1. agilo-0.9.15

    2. Babel-1.3

    3. CherryPy-3.2.2

    4. Genshi-0.7

    5. psycopg2-2.7.4

    6. pysugarcrm-1.0.0

    7. TracMenusPlugin-0.1

Is that the output of "pip list"? Unless you have an integration to Trac via a plugin, CherryPy, pysugarcrm and Django (mentioned below) have nothing to do with your Trac instance, other than possibly sharing the same Python installation.

Other than those plugins I have used some downloaded and modified plugins as well.

You can also see your installed plugins on the /about page, provided you have TRAC_ADMIN or CONFIG_VIEW permission. Looks like you have just Agilo and TracMenusPlugin installed, but please check /about for a full listing. You may have some plugins installed in the plugin directory which wouldn't be output by "pip list".

The permission policies from trac.ini are,

permission_policies = DefaultPermissionPolicy, LegacyAttachmentPolicy

That is standard, the default, so nothing unusual there.

The lines logged in that attachment are just a sample. I am seeing similar repetitions for a lot of other issues as well. They are repeated for ten to fifteen times instead of just one.

Also, my trac site is very slow. I am using Apache + mod_wsgi + Django. What may be the reason for this slowness? Could you please direct me to fix the performance?

The repeated logging may be related to the slowness. It looks like permission checks are being repeated, which can slow the response time.

Slowness could also be due to many other things. Please describe your configuration: e.g. TracStandalone or Apache with mod_wsgi or mod_python? Windows or Linux?

And please take a look at:

https://trac.edgewall.org/wiki/TracTroubleshooting

https://trac.edgewall.org/wiki/TracPerformance

- Ryan

Velu Narasimman

unread,

Nov 6, 2019, 12:08:19 AM11/6/19

to RjOllos, Trac Users

Hi,

I have used Trac1.0.15 hosted on apache2 as a virtual host. So it is like Trac + Apache2 + mod_wsgi combination. Also we have integrated DJANGO site with this Trac instance into that same virtualhost configuration.

From your previous mail, if performance checks are repeated and that will cause slowness, how can I fix that up? any idea? Also I have attached few more screenshots for your reference. It seems like the Environment is reloaded repeatedly or something strange is going on here.

In one of those screenshots "Nextnames" is a text that I printed to log file. But it is repeated many times instead of just one. Hope you can get some idea about that.

I welcome your help to fix these up!

Thank you!

Velu Narasimman | Engineer, Information Systems

Mobile: +91-44-6740 400 | Phone: +91-44-6740 4000 | Extn. 11535 |

Website: www.aspiresys.com | Blog: http://blog.aspiresys.com

India’s Best Companies to Work for 2019!

From: trac-...@googlegroups.com <trac-...@googlegroups.com> on behalf of RjOllos <rjo...@gmail.com>
Sent: Wednesday, November 6, 2019 4:50 AM

To: Trac Users <trac-...@googlegroups.com>
Subject: Re: [Trac] Why am I seeing a lot of repeated log lines in trac.log?

CAUTION: External email. Please verify the veracity of the sender/content before opening a link or an attachment.

--

You received this message because you are subscribed to the Google Groups "Trac Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/trac-users/85f5d4a1-24de-4675-8b30-b22870fabfb1%40googlegroups.com.

environment_reloaded.png

repeated_log_lines_1.png

text_repeated.png

Velu Narasimman

unread,

Nov 6, 2019, 1:10:25 AM11/6/19

to Trac Users

One more thing is that I used Ubuntu16.04 linux and Postgresql for database.

To unsubscribe from this group and stop receiving emails from it, send an email to trac-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/trac-users/85f5d4a1-24de-4675-8b30-b22870fabfb1%40googlegroups.com.

RjOllos

unread,

Nov 6, 2019, 7:31:06 PM11/6/19

to Trac Users

On Tuesday, November 5, 2019 at 9:08:19 PM UTC-8, Velu Narasimman wrote:

Hi,

    I have used Trac1.0.15 hosted on apache2 as a virtual host. So it is like Trac + Apache2 + mod_wsgi combination. Also we have integrated DJANGO site with this Trac instance into that same virtualhost configuration.

    From your previous mail, if performance checks are repeated and that will cause slowness, how can I fix that up? any idea? Also I have attached few more screenshots for your reference. It seems like the Environment is reloaded repeatedly or something strange is going on here.

    In one of those screenshots "Nextnames" is a text that I printed to log file. But it is repeated many times instead of just one. Hope you can get some idea about that.

How did you print that to the log file? Is that an Environment.log call from a plugin?

To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/trac-users/85f5d4a1-24de-4675-8b30-b22870fabfb1%40googlegroups.com.

Velu Narasimman

unread,

Nov 7, 2019, 12:07:22 AM11/7/19

to RjOllos, Trac Users

Yes you are correct. I used Environment.log.debug("some text") line to print to the log file.

Velu Narasimman | Engineer, Information Systems

Mobile: +91-44-6740 400 | Phone: +91-44-6740 4000 | Extn. 11535 |

Website: www.aspiresys.com | Blog: http://blog.aspiresys.com

India’s Best Companies to Work for 2019!

From: trac-...@googlegroups.com <trac-...@googlegroups.com> on behalf of RjOllos <rjo...@gmail.com>

Sent: Thursday, November 7, 2019 6:01 AM

To unsubscribe from this group and stop receiving emails from it, send an email to trac-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trac-users/e6a958bd-6942-4d43-8965-fe24eddb7390%40googlegroups.com.

Ryan Ollos

unread,

Nov 7, 2019, 12:18:02 AM11/7/19

to Velu Narasimman, RjOllos, Trac Users

On Wed, Nov 6, 2019 at 9:07 PM Velu Narasimman <velu.na...@aspiresys.com> wrote:

Yes you are correct. I used Environment.log.debug("some text") line to print to the log file

You did that from a terminal? Or inserted the statement into a plugin? If plugin, was it an interface such as IRequestFilter.pre_process_request or IRequestFilter.post_process_request?

- Ryan

Velu Narasimman

unread,

Nov 7, 2019, 12:36:34 AM11/7/19

to Ryan Ollos, Trac Users

I used that statement inside of a plugin. I am managing multiple Environments(projects) here. So I used the generic logging form like, self.log.debug("write this to log file") .

In some other places due to the need I imported the environment specifically like this,

from trac.env import Environment

current_env = Environment("/path/to/environment") # environment is a Project

current_env.log.debug("Some text to the log file")

I have used this second method at many places.

Velu Narasimman | Engineer, Information Systems

Mobile: +91-44-6740 400 | Phone: +91-44-6740 4000 | Extn. 11535 |

Website: www.aspiresys.com | Blog: http://blog.aspiresys.com

India’s Best Companies to Work for 2019!

From: Ryan Ollos <rjo...@gmail.com>
Sent: Thursday, November 7, 2019 10:47 AM
To: Velu Narasimman <velu.na...@aspiresys.com>
Cc: RjOllos <rjo...@gmail.com>; Trac Users <trac-...@googlegroups.com>

Subject: Re: [Trac] Why am I seeing a lot of repeated log lines in trac.log?

CAUTION: External email. Please verify the veracity of the sender/content before opening a link or an attachment.

RjOllos

unread,

Nov 13, 2019, 12:35:26 PM11/13/19

to Trac Users

On Wednesday, November 6, 2019 at 9:36:34 PM UTC-8, Velu Narasimman wrote:

    I used that statement inside of a plugin. I am managing multiple Environments(projects) here. So I used the generic logging form like, self.log.debug("write this to log file") .

    In some other places due to the need I imported the environment specifically like this,

    from trac.env import Environment

    current_env = Environment("/path/to/environment") # environment is a Project

    current_env.log.debug("Some text to the log file")

    I have used this second method at many places.

If run that code that from the command line, I assume only one log line will be written.

If that's true, then there is likely an issue with your web server configuration that is leading to the request being processed multiple times. I assume that is related to the slowness.

Sharing your web server configuration may be a good next step.

It's strange you would need to construct an Environment instance in a plugin, since it can be accessed in a Component instance as self.env. In the rare instance that it's necessary you probably want to use "open_environment" from trac.env, with use_cache=True.

https://www.edgewall.org/docs/branches-1.2-stable/html/api/trac_env.html#functions

You may want to temporarily disable your plugins and see if the duplicate logging ceases. Some of your plugins may not be implemented correctly.

- Ryan

Velu Narasimman

unread,

Nov 14, 2019, 2:36:00 AM11/14/19

to Trac Users

Hi,

I try that "open_environment" function and let you know the improvements.

Second thing, below is my virtual host configuration. Please go through it and help me to fix any wrong configuration that I have made.

#To force daemon mode
WSGIRestrictEmbedded On
<VirtualHost *:80>
ServerName subdomain.domainname.com
Redirect /  https://subdomain.domainname.com
</VirtualHost>


<VirtualHost *:443>
    ServerName subdomain.domainname.com
    Protocols h2 h2c http/1.1

    # uat conf
    WSGIScriptAlias /auth /path/to/DJANGO/WSGI/file
    WSGIDaemonProcess uat user=www-data group=www-data threads=25 python-path=/path/to/DJANGO/environment/Sourcecode
    WSGIProcessGroup uat

    SSLEngine on
    SSLCertificateFile    /path/to/cert/file
    SSLCertificateKeyFile /path/to/key/file
    SSLCertificateChainFile /path/to/chain/file

    SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
    SSLCipherSuite EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH
    SSLHonorCipherOrder On
    SSLCompression      off
    SSLSessionTickets   off

    RewriteEngine On

    <Directory "/path/to/DJANGO/source">
        <Files wsgi.py>
        Require all granted
        </Files>
    </Directory>
    Alias /static /path/to/DJANGO/source/static
    <Directory "/path/to/DJANGO/source/static">
        Require all granted
    </Directory>
    #Trac instance configuration
    WSGIScriptAlias / /usr/share/trac/trac.wsgi
   <Directory "/opt/TracProjects">
        WSGIApplicationGroup %{GLOBAL}
        Options Indexes FollowSymLinks
        Order allow,deny
        Allow from all
    <IfModule mod_authz_core.c>
        Require all granted
    </IfModule>
    </Directory>

    # Mapping Static files
    Alias /PMO/chrome /var/www/html/htdocs/htdocs
    AliasMatch  "^/([^/]*)/chrome/(.*)$" "/var/www/html/htdocs/htdocs/$2"
    <Directory "/var/www/html/htdocs/htdocs">
         Options -Indexes
        <IfModule mod_authz_core.c>
            Require all granted
        </IfModule>
    </Directory>

    # Secure headers
    <IfModule mod_headers.c>
        Header set X-Content-Type-Options "nosniff"
        Header set X-XSS-Protection  "1;mode = block"
        Header always append X-Frame-Options SAMEORIGIN
        Header always set Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
        Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure
    </IfModule>
Alias /authtimesheet "/var/www/html/A_PHP_site/public"

<Directory /var/www/html/A_PHP_site/public>
Options Indexes FollowSymLinks
DirectoryIndex index.php
AllowOverride All
Require all granted
</Directory>
 ErrorLog /var/log/apache2/error.log
 LogLevel info
 CustomLog /var/log/apache2/access.log combined
</VirtualHost>

So if you see, here I have covered a few instances all together.

1.

https://subdomain.domainname.com - This is the main site. This is mostly our Trac, implemented with Trac 1.0.15. But inside it we have hooked apps of two other frameworks.

    2. We have a DJANGO app hooked and a PHP_site(Zend framework) hooked. We hooked them with Trac by creating different menus for them and then redirecting the tracffic to those pages.

Whenever the user hits the url

https://subdomain.domainname.com/auth apache will direct user to DJANGO app. Then if the user clicks one the menu there, he will be directed to the Trac site I am running.

    For your information, I have already created a separate thread about "Mapping the static resources" for this site here

    https://groups.google.com/d/msgid/trac-users/f06818d2-701f-4904-b8cc-2051939e0719%40googlegroups.com?utm_medium=email&utm_source=footer

    Of course, I have installed my plugins globally, and I have only one single Trac instance in the server. So mapped the static resources as seen in the above virtualhost conf file. Now, tell me should I run the trac-admin deploy command for all those projects?(number is growing, for now 1000+ projects).

    Second how can I change the conf file to make the site faster? Does this configuration affect logging the same lines multiple times?

    I welcome all your suggestions to improve this site! Thank you in advance.

RjOllos

unread,

Nov 16, 2019, 11:15:02 AM11/16/19

to Trac Users

On Wednesday, November 13, 2019 at 11:36:00 PM UTC-8, Velu Narasimman wrote:

Of course, I have installed my plugins globally, and I have only one single Trac instance in the server. So mapped the static resources as seen in the above virtualhost conf file. Now, tell me should I run the trac-admin deploy command for all those projects?(number is growing, for now 1000+ projects).

Same answer as here:

https://groups.google.com/d/msg/trac-users/2btVAoymXGw/f0lByTbDAgAJ

If all Trac sites have the same plugins, then you can run deploy for just one environment and use the extracted static resources for all of your projects. You only need to run deploy once.

If your sites share the same Python install, then your environments will share the same plugins as long as you don't install plugins to the environment plugins directory. Instead, they must be installed in a shared plugins directory or installed in Python's site-packages, and it sounds like you've done that.

Second how can I change the conf file to make the site faster? Does this configuration affect logging the same lines multiple times?

Nothing in the configuration is causing this. It's probably due to the way one of your plugins is implemented. Your site sounds very complex, and it's going to be difficult to give guidance.

- Ryan

RjOllos

unread,

Nov 16, 2019, 11:49:40 AM11/16/19

to Trac Users

I would proceed in the following two ways:

On Wednesday, November 13, 2019 at 9:35:26 AM UTC-8, RjOllos wrote:

It's strange you would need to construct an Environment instance in a plugin, since it can be accessed in a Component instance as self.env. In the rare instance that it's necessary you probably want to use "open_environment" from trac.env, with use_cache=True.
https://www.edgewall.org/docs/branches-1.2-stable/html/api/trac_env.html#functions

1. Make the suggested "open_environment" change to your plugins. Share code snippets here is you are unsure how to make the changes.

You may want to temporarily disable your plugins and see if the duplicate logging ceases. Some of your plugins may not be implemented correctly.

2. If duplicate logging persists and the site is still slow, schedule some downtime for your site. During the downtime, disable plugins one-at-a-time, and check the duplicate logging and site load-time to see if the issues are resolved. You may lose site functionality along the way, but no bother as this is just a debugging step to isolate behavior. If you determine which plugin causes the issue, please share it here and we can give guidance as to how to improve it.

- Ryan

Ryan Ollos

unread,

Nov 16, 2019, 11:57:35 AM11/16/19

to Trac Users

On Wed, Nov 13, 2019 at 11:36 PM Velu Narasimman <velava...@gmail.com> wrote:

Hi,

I try that "open_environment" function and let you know the improvements.

Second thing, below is my virtual host configuration. Please go through it and help me to fix any wrong configuration that I have made.

#Trac instance configuration WSGIScriptAlias / /usr/share/trac/trac.wsgi <Directory "/opt/TracProjects"> WSGIApplicationGroup %{GLOBAL} Options Indexes FollowSymLinks Order allow,deny Allow from all <IfModule mod_authz_core.c> Require all granted </IfModule> </Directory> # Mapping Static files Alias /PMO/chrome /var/www/html/htdocs/htdocs AliasMatch "^/([^/]*)/chrome/(.*)$" "/var/www/html/htdocs/htdocs/$2" <Directory "/var/www/html/htdocs/htdocs"> Options -Indexes <IfModule mod_authz_core.c> Require all granted </IfModule> </Directory>

Looks like the above is the only Trac-specific part of your configuration.

You should be able to delete the "Alias" directive as the "AliasMatch" directive will work for path /PMO/Chrome.

I had a similar scenario, with 3 environments all sharing the same Python instance and thus same Trac install, and all plugins installed to "site-packages":

/var/trac/env1

/var/trac/env2

/var/trac/env3

Looks like you have pretty much the same situation.

I ran "trac-admin deploy /var/trac/env1 /var/trac/www" and then pointed the Apache Alias and WSGIScriptAlias to this directory.

Alias points to /var/trac/www/htdocs, WSGIScriptAlias points to /var/trac/www/cgi-bin/trac.wsgi

Of course, since all environments are sharing the same trac.wsgi, I believe you must set trac.env_path. Take a look at the code in the trac.wsgi that was created by the deploy command. For my case, it looks like this:

def application(environ, start_request):
if not 'trac.env_parent_dir' in environ:
environ.setdefault('trac.env_path', '/var/trac/env1')
if 'PYTHON_EGG_CACHE' in environ:
os.environ['PYTHON_EGG_CACHE'] = environ['PYTHON_EGG_CACHE']
elif 'trac.env_path' in environ:
os.environ['PYTHON_EGG_CACHE'] = \
os.path.join(environ['trac.env_path'], '.egg-cache')
elif 'trac.env_parent_dir' in environ:
os.environ['PYTHON_EGG_CACHE'] = \
os.path.join(environ['trac.env_parent_dir'], '.egg-cache')
from trac.web.main import dispatch_request
return dispatch_request(environ, start_request)

Sharing the same trac.wsgi for all projects would result in trac.env_path being set to /var/trac/env1 for all projects. That is, unless you set trac.env_path or trac.env_parent_dir in the Apache config using:

SetEnv trac.env_path /var/trac/$envname

or

SetEnv trac.env_parent_dir /var/trac

The latter is fine if all of your environments are in the same directory.

Looking at your config snippet:

WSGIScriptAlias / /usr/share/trac/trac.wsgi
<Directory "/opt/TracProjects">
WSGIApplicationGroup %{GLOBAL}
Options Indexes FollowSymLinks
Order allow,deny
Allow from all
<IfModule mod_authz_core.c>
Require all granted
</IfModule>
</Directory>

Looks like your Trac site is served at path / and all your Trac environments are under /opt/TracProjects.

It looks like your deploy execution created /var/www/html/htdocs/htdocs, so I assume also /var/www/html/htdocs/cgi-bin/trac.wsgi was created.

I would use something like this (untested, so you may have to correct some typos). Assumes Apache 2.4.

WSGIScriptAlias / /var/www/html/htdocs/cgi-bin/trac.wsgi

<Directory /var/www/html/htdocs/cgi-bin>
WSGIApplicationGroup %{GLOBAL}
Require all granted
</Directory>

SetEnv trac.env_parent_dir /opt/TracProjects

</Location>

AliasMatch "^/([^/]*)/chrome/(.*)$" "/var/www/html/htdocs/htdocs/$2"
<Directory "/var/www/html/htdocs/htdocs">
Options -Indexes

Require all granted

</Directory>

You ran deploy with "/var/www/html/htdocs" as the argument?

Might be cleaner to run "trac-admin /path/to/env deploy /var/www/trac" and create:

/var/www/trac/htdocs

/var/www/trac/cgi-bin

You'd have to adjust paths above if you make these path changes.

I hope it makes sense.

- Ryan

Reply all

Reply to author

Forward