How to run test suite in Evergreen ?

84 views
Skip to first unread message

bai...@gmail.com

unread,
Jan 10, 2021, 7:18:17 AM1/10/21
to wiredtiger-users
Hi. 

I try to find docs about how to run test suite in wiredtiger source tree with Evergreen, but I found in http://source.wiredtiger.com/develop/testing.html, the doc is out-dated, at least Jenkins is not used, right ?

Anyone who can share what instructions are about this ? How to launch evergreen web ui quickly, how to install wiredtiger python library, and so on. I don't see direct steps on evergreen wiki.

Luke Chen

unread,
Jan 10, 2021, 7:50:20 PM1/10/21
to wiredtig...@googlegroups.com
Hi there, 

Thanks for your interest to wiredtiger testing. 

Evergreen is our major CI system these days, though we still have some tests running on Jenkins to achieve a complete test coverage.

When a pull request (PR) is raised on Github, an Evergreen patch build will be automatically created to run a set of tests for sanity check. One of our admins will help with triggering the testing if a PR is raised by an external contributor. 

We have a wiki page describing how to run Python test suite on a Linux machine. If you run into any issue following the steps on the page please feel free to let us know, we are more than happy to help. 

Luke

--
You received this message because you are subscribed to the Google Groups "wiredtiger-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wiredtiger-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wiredtiger-users/e3c217a0-71d2-4ee8-a00d-b106c394491dn%40googlegroups.com.

bai...@gmail.com

unread,
Jan 10, 2021, 10:32:47 PM1/10/21
to wiredtiger-users
Hi. Luke

Thansk for guidance.  I tried, but met with the following error:
root@wiredtiger ((mongodb-4.4.1) *%)$ env LD_LIBRARY_PATH=".libs" python3 ./test/suite/run.py test/suite/test_base01.py 
E
======================================================================
ERROR: test/suite/test_base01 (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test/suite/test_base01
Traceback (most recent call last):
  File "/usr/lib64/python3.6/unittest/loader.py", line 153, in loadTestsFromName
    module = __import__(module_name)
ModuleNotFoundError: No module named 'test/suite/test_base01'


----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

I installed wiredtiger to /opt/wt-10.0 (configure command is: CFLAGS="-g -O0" ./configure --enable-snappy --enable-python --prefix=/opt/wt-10.0)

wiredtiger ((mongodb-4.4.1) *%)$ ls /opt/wt-10.0/lib
libwiredtiger-10.0.0.so  libwiredtiger.a  libwiredtiger.la  libwiredtiger_snappy.a  libwiredtiger_snappy.la  libwiredtiger_snappy.so  libwiredtiger.so  pkgconfig

The wt python library was installed to /usr/local/lib64/python3.6,

a4.png

Anything wrong ?

bai...@gmail.com

unread,
Jan 10, 2021, 10:54:26 PM1/10/21
to wiredtiger-users
If I do this, there's the following error: 

a5.png

So how should I run a single py ?

Luke Chen

unread,
Jan 10, 2021, 11:25:03 PM1/10/21
to wiredtig...@googlegroups.com
Can you try using only the test file name (instead of the full path) for the last argument
e.g. $ python3 ./test/suite/run.py test_base01.py
or even $ python3 ./test/suite/run.py test_base01
The testing framework will regex the input (value of the last argument) to figure out the right set of test files/classes. 

Also, the wiki page assumes you "make" under the "build_posix" directory and generate all library files under "build_posix/.libs" folder. 
In your case, it looks the library files were generated under /opt/wt-10.0/lib. So you may try prefixing env LD_LIBRARY_PATH="lib" for the python command if you run it under /opt/wt-10.0

bai...@gmail.com

unread,
Jan 10, 2021, 11:43:50 PM1/10/21
to wiredtiger-users
Actually I tried, (I thought the libraries in .libs are same as in /opt/wt-10.0/lib, so I haven't uploaded the case), 
a6.png

Luke Chen

unread,
Jan 11, 2021, 12:55:21 AM1/11/21
to wiredtig...@googlegroups.com, Donald Anderson
This error is actually different from the initial error you reported. I'm not able to reproduce it locally. 

Can you confirm the OS variant/version, SWIG version, and Python version that you used in your environment? 

Don,

Any idea what could be possibly wrong that generated the argument TypeError? 

Luke

bai...@gmail.com

unread,
Jan 11, 2021, 1:49:11 AM1/11/21
to wiredtiger-users
OS is like redhat , kernel version 4.14
Python3 : 3.6.8
swig: 4.0.2

Is is becasue swig's version is too new ?

bai...@gmail.com

unread,
Jan 11, 2021, 2:09:49 AM1/11/21
to wiredtiger-users

Donald Anderson

unread,
Jan 11, 2021, 9:33:04 AM1/11/21
to wiredtig...@googlegroups.com
Hello,

It looks like WT-6091 may be the problem here.  The workaround for WT-6091 is to remove lang/python/wiredtiger_wrap.c (from the top level of the tree, it is not relative to build_posix).  Then run make again.

Which version or branch of wiredtiger are you building?  I was not aware that this can still happen in the develop branch.

- Don

Don Anderson


bai...@gmail.com

unread,
Jan 11, 2021, 8:59:51 PM1/11/21
to wiredtiger-users
Wow, I tried this and worked !

I'm on the mongodb-4.4.1 tag, 579966149b9094922f22d9bd7e8f3b8b9db1ccf7

bai...@gmail.com

unread,
Jan 11, 2021, 9:24:46 PM1/11/21
to wiredtiger-users
Thanks to Luke and Don.

Another thing, I compile Evergreen and see the output of `evergreen -h` (many commands). I want to run it on local machine, is that possible ? (I've noted that there're evergreen.yml under wiredtiger source tree.)
 I didn't see steps about how to make use of those commands from https://github.com/evergreen-ci/evergreen/wiki

Luke Chen

unread,
Jan 11, 2021, 10:41:14 PM1/11/21
to wiredtig...@googlegroups.com
It's great to hear the workaround solved your issue. 

With regard to Evergreen, I guess you are attempting to create a patch build manually using the CLI. You will need an Evergreen credential to be able to use the CLI. 
The Evergreen infrastructure is maintained internally by the MongoDB team, and I doubt it currently opens access to community members. 

With that being said, Evergreen (and any CI system) just provides an automatic way of running tests. In most cases one should be able to run those tests manually without relying on Evergreen. If you have any specific need e.g. to access some Evergreen test failure log or repro a failure, I'm sure one of us could offer helps. 

Luke

bai...@gmail.com

unread,
Jan 11, 2021, 11:10:01 PM1/11/21
to wiredtiger-users
Good to know. I see this article https://engineering.mongodb.com/post/evergreen-continuous-integration-why-we-reinvented-the-wheel , and I consider it to be a fantastic CI system ! So I'm kind of curious. 
Reply all
Reply to author
Forward
0 new messages