Re: A script to simplify writing Valgrind suppressions to keep the memory waterfall green

140 views
Skip to first unread message

Timur Iskhodzhanov

unread,
Aug 4, 2010, 5:18:35 PM8/4/10
to chromium-dev, cben...@chromium.org, glider, hb...@chromium.org, jhaw...@chromium.org, joc...@chromium.org, osh...@chromium.org, stuart...@chromium.org, thestig, k...@chromium.org, Nicolas Sylvain, Marc-Antoine Ruel
Hi,

After filing a few dozens of Valgrind suppressions to keep the memory waterfall green I've started thinking there should be some easier and more effective way to do it.

What I disliked the most was that once a suppression is commited, one had to wait 2-4 hours for the current build cycle to end and one more cycle with the new suppression.
Since we have three different platforms which can have the same report with subtle differences in mangling (I hate mangling btw), this complicates stuff a lot and sometimes it's quite a waste of time for the suppression writers.

Recently I've created a script to simplify this routine and we've (me and glider@) tried it with good results.

You can give it a try:
./tools/valgrind/waterfall.sh fetch|match
fetch: download memory waterfall logs for those builds which have failed Valgrind steps 
 (usually that means there is an unsuppressed Valgrind report)
Nicolas, Marc-Antoine, I hope the fetch doesn't DDoS the waterfall. At least it downloads almost the same amount of logs a Valgrind sheriff would download manually to get the reports. Probably we can optimize here if needed.
match: list all the fetched reports that don't match local suppressions

Typical workflow is as follows:
./tools/valgrind/waterfall.sh fetch
svn up tools/valgrind/memcheck # get your suppressions in sync
./tools/valgrind/waterfall.sh match # get the list of reports
<write new suppressions or generalize existing>
./tools/valgrind/waterfall.sh match # make sure the number of reports decreased
[optional: generalize the new suppression to match more reports - vimdiff is your friend]
./tools/valgrind/waterfall.sh match # make sure the number of reports decreased even more
<file a bug>
<commit a suppression>

Your feedback and patches are welcome!
Timur Iskhodzhanov,
Google Russia


 

Jochen Eisinger

unread,
Aug 11, 2010, 9:39:33 AM8/11/10
to Timur Iskhodzhanov, chromium-dev, cben...@chromium.org, glider, hb...@chromium.org, jhaw...@chromium.org, osh...@chromium.org, stuart...@chromium.org, thestig, k...@chromium.org, Nicolas Sylvain, Marc-Antoine Ruel
I've added an command "blame" to this script. "waterfall.sh match"
will now also output the error hashes. With a given hash, you can
search for the first run when this specific suppression was observed
on a specific builder and output the blame list for that revision:

waterfall.sh blame <bot> <test> <hash>

e.g.

tools/valgrind/waterfall.sh blame "Linux Tests (valgrind)(3)" ui 152F078A
Searching for latest revision without this hash........
Hash #152F078A# occurs in build #6327 for the first time.
Receiving blame list.
Revision: 55620
Blamelist:
...

Please use this command with care, since it hammers the waterfall quite a bit.

-jochen

Marc-Antoine Ruel

unread,
Sep 27, 2010, 3:23:14 PM9/27/10
to Timur Iskhodzhanov, chromium-dev, cben...@chromium.org, glider, hb...@chromium.org, jhaw...@chromium.org, joc...@chromium.org, osh...@chromium.org, stuart...@chromium.org, thestig, k...@chromium.org, Nicolas Sylvain
Sorry I had skipped over this thread, I wasn't in the cc list.

Please take a look at http://build.chromium.org/buildbot/memory/json/help and probably think about rewriting it in python as bash scripts hardly scale. Each builders have the cached_builds property which lists the builds that won't represent a performance issue on the build master to access so it's good if you can limit your scan to these. Otherwise just insert a time.sleep(1) or slightly more between the url fetch.

M-A
Reply all
Reply to author
Forward
0 new messages