Quick Question: do tests fail on macos?

39 views
Skip to first unread message

Mayur R

unread,
Apr 22, 2022, 4:16:03 PM4/22/22
to Prometheus Developers
make build succeeds then I did following

MacBook-Pro-2:prometheus mraleras$ make test
>> running all tests
GO111MODULE=on go test -race  ./...
ok      github.com/prometheus/prometheus/cmd/prometheus 111.481s
ok      github.com/prometheus/prometheus/cmd/promtool   130.848s
ok      github.com/prometheus/prometheus/config 2.780s
ok      github.com/prometheus/prometheus/discovery      5.684s
?       github.com/prometheus/prometheus/discovery/aws  [no test files]
ok      github.com/prometheus/prometheus/discovery/azure        0.515s
ok      github.com/prometheus/prometheus/discovery/consul       3.499s
ok      github.com/prometheus/prometheus/discovery/digitalocean 1.938s
ok      github.com/prometheus/prometheus/discovery/dns  1.441s
ok      github.com/prometheus/prometheus/discovery/eureka       1.135s
ok      github.com/prometheus/prometheus/discovery/file 3.167s
?       github.com/prometheus/prometheus/discovery/gce  [no test files]
ok      github.com/prometheus/prometheus/discovery/hetzner      0.650s
ok      github.com/prometheus/prometheus/discovery/http 1.572s
?       github.com/prometheus/prometheus/discovery/install      [no test files]
ok      github.com/prometheus/prometheus/discovery/kubernetes   27.412s
ok      github.com/prometheus/prometheus/discovery/legacymanager        3.735s
ok      github.com/prometheus/prometheus/discovery/linode       0.618s
ok      github.com/prometheus/prometheus/discovery/marathon     0.368s
ok      github.com/prometheus/prometheus/discovery/moby 6.517s
ok      github.com/prometheus/prometheus/discovery/openstack    0.521s
ok      github.com/prometheus/prometheus/discovery/puppetdb     1.199s
ok      github.com/prometheus/prometheus/discovery/refresh      0.387s
ok      github.com/prometheus/prometheus/discovery/scaleway     1.810s
ok      github.com/prometheus/prometheus/discovery/targetgroup  0.261s
ok      github.com/prometheus/prometheus/discovery/triton       0.494s
ok      github.com/prometheus/prometheus/discovery/uyuni        0.532s
ok      github.com/prometheus/prometheus/discovery/xds  2.572s
--- FAIL: TestNewDiscoveryError (0.07s)
    zookeeper_test.go:35: expected error, got nil
FAIL
FAIL    github.com/prometheus/prometheus/discovery/zookeeper    0.369s
ok      github.com/prometheus/prometheus/documentation/examples/custom-sd/adapter       0.591s
?       github.com/prometheus/prometheus/documentation/examples/custom-sd/adapter-usage [no test files]
?       github.com/prometheus/prometheus/model/exemplar [no test files]
ok      github.com/prometheus/prometheus/model/labels   0.370s
ok      github.com/prometheus/prometheus/model/relabel  0.251s
ok      github.com/prometheus/prometheus/model/rulefmt  1.043s
ok      github.com/prometheus/prometheus/model/textparse        0.395s
?       github.com/prometheus/prometheus/model/timestamp        [no test files]
?       github.com/prometheus/prometheus/model/value    [no test files]
ok      github.com/prometheus/prometheus/notifier       0.955s
?       github.com/prometheus/prometheus/plugins        [no test files]
?       github.com/prometheus/prometheus/prompb [no test files]
ok      github.com/prometheus/prometheus/promql 29.842s
ok      github.com/prometheus/prometheus/promql/parser  8.617s
ok      github.com/prometheus/prometheus/rules  44.925s
ok      github.com/prometheus/prometheus/scrape 20.093s
ok      github.com/prometheus/prometheus/storage        1.526s
ok      github.com/prometheus/prometheus/storage/remote 14.953s
ok      github.com/prometheus/prometheus/template       0.438s
ok      github.com/prometheus/prometheus/tracing        0.516s
level=info msg="Replaying on-disk memory mappable chunks if any"
level=info msg="On-disk memory mappable chunks replay completed" duration=4.315µs
level=info msg="Replaying WAL, this may take a while"
level=warn msg="Unknown series references" samples=3803 exemplars=0
level=info msg="WAL segment loaded" segment=0 maxSegment=1
level=info msg="WAL segment loaded" segment=1 maxSegment=1
level=info msg="WAL replay completed" checkpoint_replay_duration=235.938µs wal_replay_duration=2.655315502s total_replay_duration=2.655577365s
level=info msg="Replaying on-disk memory mappable chunks if any"
level=info msg="On-disk memory mappable chunks replay completed" duration=10.581575ms
level=info msg="Replaying WAL, this may take a while"
level=warn msg="Unknown series references" samples=6473 exemplars=0
level=info msg="WAL segment loaded" segment=0 maxSegment=1
level=info msg="WAL segment loaded" segment=1 maxSegment=1
level=info msg="WAL replay completed" checkpoint_replay_duration=376.876µs wal_replay_duration=2.77993431s total_replay_duration=2.790944711s
panic: test timed out after 10m0s

goroutine 9479 [running]:
testing.(*M).startAlarm.func1()
        /usr/local/Cellar/go/1.17.8/libexec/src/testing/testing.go:1788 +0xbb
created by time.goFunc
        /usr/local/Cellar/go/1.17.8/libexec/src/time/sleep.go:180 +0x4a

goroutine 1 [chan receive, 7 minutes]:
testing.(*T).Run(0xc0002d4d00, {0x1a6320c, 0x25}, 0x1a82618)
        /usr/local/Cellar/go/1.17.8/libexec/src/testing/testing.go:1307 +0x752
testing.runTests.func1(0x0)
        /usr/local/Cellar/go/1.17.8/libexec/src/testing/testing.go:1598 +0x9a
testing.tRunner(0xc0002d4d00, 0xc00033fa60)
        /usr/local/Cellar/go/1.17.8/libexec/src/testing/testing.go:1259 +0x230
testing.runTests(0xc0002c2280, {0x2a75bc0, 0x96, 0x96}, {0x203000, 0x19395e2, 0x2a7b700})
        /usr/local/Cellar/go/1.17.8/libexec/src/testing/testing.go:1596 +0x7cb
testing.(*M).Run(0xc0002c2280)
        /usr/local/Cellar/go/1.17.8/libexec/src/testing/testing.go:1504 +0x9d2
go.uber.org/goleak.VerifyTestMain({0x25b9cc0, 0xc0002c2280}, {0xc0000b5e70, 0x2, 0x2})
        /Users/mraleras/go/pkg/mod/go.uber.org/gol...@v1.1.12/testmain.go:53 +0x5a
github.com/prometheus/prometheus/tsdb.TestMain(0x480)
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/db_test.go:63 +0x372
main.main()
        _testmain.go:385 +0x265

goroutine 6659 [semacquire]:
sync.runtime_Semacquire(0xc0004e0204)
        /usr/local/Cellar/go/1.17.8/libexec/src/runtime/sema.go:56 +0x25
sync.(*WaitGroup).Wait(0xc0004e0204)
        /usr/local/Cellar/go/1.17.8/libexec/src/sync/waitgroup.go:130 +0xea
github.com/prometheus/prometheus/tsdb.TestTombstoneCleanRetentionLimitsRace(0xc001942b60)
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/db_test.go:1310 +0x306
testing.tRunner(0xc001942b60, 0x1a82618)
        /usr/local/Cellar/go/1.17.8/libexec/src/testing/testing.go:1259 +0x230
created by testing.(*T).Run
        /usr/local/Cellar/go/1.17.8/libexec/src/testing/testing.go:1306 +0x727

goroutine 9047 [select]:
github.com/prometheus/prometheus/tsdb.(*DB).run(0xc0001ea270)
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/db.go:811 +0x2ef
created by github.com/prometheus/prometheus/tsdb.open
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/db.go:758 +0x1a78

goroutine 8935 [select]:
github.com/prometheus/prometheus/tsdb/wal.(*WAL).run(0xc0002ee000)
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/wal/wal.go:332 +0x129
created by github.com/prometheus/prometheus/tsdb/wal.NewSize
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/wal/wal.go:301 +0x531

goroutine 9455 [syscall]:
syscall.syscall(0x10c9aa0, 0xc000114000, 0x1000000, 0x0)
        /usr/local/Cellar/go/1.17.8/libexec/src/runtime/sys_darwin.go:22 +0x3b
syscall.Open({0xc000066320, 0x9d}, 0x2a7a8a0, 0x0)
        /usr/local/Cellar/go/1.17.8/libexec/src/syscall/zsyscall_darwin_amd64.go:1088 +0x10e
os.openFileNolog({0xc000066320, 0x9d}, 0x0, 0x0)
        /usr/local/Cellar/go/1.17.8/libexec/src/os/file_unix.go:217 +0x125
os.OpenFile({0xc000066320, 0x9d}, 0x0, 0x400000)
        /usr/local/Cellar/go/1.17.8/libexec/src/os/file.go:338 +0x5d
os.Open(...)
        /usr/local/Cellar/go/1.17.8/libexec/src/os/file.go:318
github.com/prometheus/prometheus/tsdb/fileutil.OpenMmapFileWithSize({0xc000066320, 0x9d}, 0x0)
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/fileutil/mmap.go:32 +0x7d
github.com/prometheus/prometheus/tsdb/fileutil.OpenMmapFile(...)
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/fileutil/mmap.go:28
github.com/prometheus/prometheus/tsdb/index.(*Writer).writeLabelIndices(0xc0000ffa20)
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/index/index.go:577 +0xe5
github.com/prometheus/prometheus/tsdb/index.(*Writer).ensureStage(0xc0000ffa20, 0x3)
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/index/index.go:382 +0x1b6
github.com/prometheus/prometheus/tsdb/index.(*Writer).Close(0xc0000ffa20)
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/index/index.go:1027 +0x3f
github.com/prometheus/prometheus/tsdb.(*LeveledCompactor).write(0xc001b56360, {0xc0000262a0, 0x64}, 0xc0001770e0, {0xc000131aa8, 0x1, 0x1})
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/compact.go:613 +0x11c7
github.com/prometheus/prometheus/tsdb.(*LeveledCompactor).Write(0xc001b56360, {0xc0000262a0, 0x64}, {0x25cb178, 0xc0000e72c0}, 0xe, 0xf, 0xc000177050)
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/compact.go:504 +0x446
github.com/prometheus/prometheus/tsdb.(*Block).CleanTombstones(0xc0000e72c0, {0xc0000262a0, 0x64}, {0x25c2800, 0xc001b56360})
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/block.go:606 +0x29c
github.com/prometheus/prometheus/tsdb.(*DB).CleanTombstones(0xc0001ea270)
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/db.go:1684 +0x3bd
github.com/prometheus/prometheus/tsdb.TestTombstoneCleanRetentionLimitsRace.func2()
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/db_test.go:1307 +0xd8
created by github.com/prometheus/prometheus/tsdb.TestTombstoneCleanRetentionLimitsRace
        /Users/mraleras/go/src/github.com/prometheus/prometheus/tsdb/db_test.go:1304 +0x2f9
FAIL    github.com/prometheus/prometheus/tsdb   600.490s
ok      github.com/prometheus/prometheus/tsdb/agent     33.503s
ok      github.com/prometheus/prometheus/tsdb/chunkenc  0.721s
ok      github.com/prometheus/prometheus/tsdb/chunks    36.631s
?       github.com/prometheus/prometheus/tsdb/encoding  [no test files]
?       github.com/prometheus/prometheus/tsdb/errors    [no test files]
ok      github.com/prometheus/prometheus/tsdb/fileutil  0.597s
ok      github.com/prometheus/prometheus/tsdb/goversion 0.318s
ok      github.com/prometheus/prometheus/tsdb/index     83.811s
ok      github.com/prometheus/prometheus/tsdb/record    0.706s
ok      github.com/prometheus/prometheus/tsdb/test      0.602s [no tests to run]
ok      github.com/prometheus/prometheus/tsdb/tombstones        0.634s
ok      github.com/prometheus/prometheus/tsdb/tsdbutil  0.323s
ok      github.com/prometheus/prometheus/tsdb/wal       70.393s
?       github.com/prometheus/prometheus/util/gate      [no test files]
ok      github.com/prometheus/prometheus/util/httputil  0.504s
ok      github.com/prometheus/prometheus/util/logging   0.784s
?       github.com/prometheus/prometheus/util/osutil    [no test files]
ok      github.com/prometheus/prometheus/util/pool      0.448s
?       github.com/prometheus/prometheus/util/runtime   [no test files]
ok      github.com/prometheus/prometheus/util/stats     0.747s
ok      github.com/prometheus/prometheus/util/strutil   0.846s
?       github.com/prometheus/prometheus/util/teststorage       [no test files]
?       github.com/prometheus/prometheus/util/testutil  [no test files]
?       github.com/prometheus/prometheus/util/treecache [no test files]
ok      github.com/prometheus/prometheus/web    17.538s
ok      github.com/prometheus/prometheus/web/api/v1     4.295s
?       github.com/prometheus/prometheus/web/ui [no test files]
FAIL
make: *** [common-test] Error 1
MacBook-Pro-2:prometheus mraleras$ 

is this expected on macos Monterey 12.2.1, Golang version 1.17.8?
I am on main branch with commit-id fc3f1b8ded810a1263f72d37db9db4e8ce726120

with no changes of my own on top it.




Matthias Rampke

unread,
May 6, 2022, 12:01:04 PM5/6/22
to Mayur R, Prometheus Developers
Hi,

I'm trying this out – I can't reproduce the zookeeper failure (on main as of today) but I ran into the same problem on the TSDB test.

I think what's happening is that the test actually takes too long. The test loops 20 times, and in each loop writes out 20 blocks. I don't know what macOS is doing there, it takes >1s on my (not old, not new) macbook to write one block.

I don't know if there is a more efficient way to run this test on macOS (or to speed up tsdb on macOS in general?), but I think in the meantime I think we can avoid hitting the timeout.

/MR

--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/c8f0e38e-495e-4224-be16-5c3d5ac2633fn%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages