"WARNING: could not attach metadata for @simd loop" on Travis

[René Donner]

Hi,

updated my package today, and am suddenly getting a lot of

WARNING: could not attach metadata for @simd loop

messages on Travis - it also seems to slow down Julia dramatically, as Travis in the end timeouts after 12min. It only occurs on the nightly build (0.4pre), release finishes in 1.5 minutes:

https://travis-ci.org/rened/FunctionalDataUtils.jl/jobs/73909060

The tests work locally for me on OSX and Linux (64bit) on today's master.

Has anybody seen similar things? How can I perform (something like) a git bisect on Travis?

Thanks,

Rene

[Tony Kelman]

This isn't integrated in a direct way into the Travis `language: julia` setup right now, but we do keep old nightlies around for about a month at https://s3.amazonaws.com/julianightlies/ so you could manually download and extract a range of those to see when the problem started happening.

[Matt Bauman]

That warning occurs when inlining is turned off.  Pkg.test turns off inlining when called with `coverage=true`.  This is so the coverage metrics don't miss small functions that always get inlined.  It also definitely causes things to run a lot slower.

You can try running your tests locally with `--inline=no`, and see how long it takes to complete, which may give you a sense of where Travis is getting stuck.

[Simon Kornblith]

See also https://github.com/JuliaLang/julia/issues/10774, which would in most cases eliminate the warning and make performance more reasonable.

[René Donner]

Thanks you all very much for your input!

Indeed, turning off inlining fixed both the @simd warnings and the timeout.

I did this by adding

script:
- julia -e 'Pkg.init(); Pkg.clone("FunctionalDataUtils")'
- julia -e 'Pkg.test("FunctionalDataUtils")'

to the default (i.e. "language: julia" at the top) travis file.

Is there a cleaner way for this, i.e. a YAML parameter like "coverage: false"?

Thanks!

[Tony Kelman]

Not currently, but there maybe could be. We decided to enable it by default several months ago since we figured that was the slightly more useful choice and weren't expecting to to cause too serious performance issues. I think you're the first counterexample to that I've heard about. It might be worth looking into what makes your packages' test timings so highly susceptible to inlining, to see whether or not the behavior is expected.

[René Donner]

I have identified the two hotspots where runtime is 50-100x slower without inlining.

Examples, with inlining:

@time sha256(reinterpret(Uint8,vec(zeros(1000,10000))))
0.716542 seconds (1.25 M allocations: 171.667 MB, 0.40% gc time)

@time fit(PCA,rand(2,100000))
0.019834 seconds (88 allocations: 3.056 MB, 91.68% gc time)

Without inlining:

@time sha256(reinterpret(Uint8,vec(zeros(1000,10000))))
82.564995 seconds (1.24 G allocations: 18.645 GB, 1.31% gc time)

@time fit(PCA,rand(2,100000))
0.849904 seconds (9.50 M allocations: 218.149 MB, 2.98% gc time)

I don't think there is anything wrong with this, I just happen so rely on this in my tests, causing the timeouts. I'll simply set coverage=false.

Thanks again!

Am 04.08.2015 um 08:59 schrieb Tony Kelman <to...@kelman.net>:

> Not currently, but there maybe could be. We decided to enable it by default several months ago since we figured that was the slightly more useful choice and weren't expecting to to cause too serious performance issues. I think you're the first counterexample to that I've heard about. It might be worth looking into what makes your packages' test timings so highly susceptible to inlining, to see whether or not the behavior is expected.
>
>
> On Monday, August 3, 2015 at 11:51:56 PM UTC-7, rene wrote:
> Thanks you all very much for your input!
>
> Indeed, turning off inlining fixed both the @simd warnings and the timeout.
>
> I did this by adding
>
> script:
> - julia -e 'Pkg.init(); Pkg.clone("FunctionalDataUtils")'
> - julia -e 'Pkg.test("FunctionalDataUtils")'
>
> to the default (i.e. "language: julia" at the top) travis file.
>
> Is there a cleaner way for this, i.e. a YAML parameter like "coverage: false"?
>
> Thanks!
>
>
> Am 03.08.2015 um 23:34 schrieb Simon Kornblith <si...@simonster.com>:
>
> > See also https://github.com/JuliaLang/julia/issues/10774, which would in most cases eliminate the warning and make performance more reasonable.
> >
> > On Monday, August 3, 2015 at 12:42:41 PM UTC-4, Matt Bauman wrote:
> > That warning occurs when inlining is turned off. Pkg.test turns off inlining when called with `coverage=true`. This is so the coverage metrics don't miss small functions that always get inlined. It also definitely causes things to run a lot slower.
> >
> > You can try running your tests locally with `--inline=no`, and see how long it takes to complete, which may give you a sense of where Travis is getting stuck.
> >
> > On Monday, August 3, 2015 at 12:10:15 PM UTC-4, Tony Kelman wrote:

> > This isn't integrated in a direct way into the Travis `language: julia` setup right now, but we do keep old nightlies around for about a month athttps://s3.amazonaws.com/julianightlies/ so you could manually download and extract a range of those to see when the problem started happening.

[Tim Holy]

Do you need such large arrays (1000x10000), or would smaller arrays (10x100)
work equally well for your test?

--Tim

[René Donner]

Unfortunately, the large size is part of the test. What I have below is just a simplified example.

I could try to rework it, but as long as I don't set up Coveralls I don't think there is a drawback to just disabling coverage?

[Tim Holy]

That's fine, just thought I'd ask.

--Tim