How to run test suite in Evergreen ?

[bai...@gmail.com]

Hi. 

I try to find docs about how to run test suite in wiredtiger source tree with Evergreen, but I found in http://source.wiredtiger.com/develop/testing.html, the doc is out-dated, at least Jenkins is not used, right ?

Anyone who can share what instructions are about this ? How to launch evergreen web ui quickly, how to install wiredtiger python library, and so on. I don't see direct steps on evergreen wiki.

[Luke Chen]

Hi there, 

Thanks for your interest to wiredtiger testing. 

Evergreen is our major CI system these days, though we still have some tests running on Jenkins to achieve a complete test coverage.

When a pull request (PR) is raised on Github, an Evergreen patch build will be automatically created to run a set of tests for sanity check. One of our admins will help with triggering the testing if a PR is raised by an external contributor. 

We have a wiki page describing how to run Python test suite on a Linux machine. If you run into any issue following the steps on the page please feel free to let us know, we are more than happy to help. 

Luke

--
You received this message because you are subscribed to the Google Groups "wiredtiger-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wiredtiger-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wiredtiger-users/e3c217a0-71d2-4ee8-a00d-b106c394491dn%40googlegroups.com.

[bai...@gmail.com]

Hi. Luke

Thansk for guidance.  I tried, but met with the following error:

root@wiredtiger ((mongodb-4.4.1) *%)$ env LD_LIBRARY_PATH=".libs" python3 ./test/suite/run.py test/suite/test_base01.py 

E

======================================================================

ERROR: test/suite/test_base01 (unittest.loader._FailedTest)

----------------------------------------------------------------------

ImportError: Failed to import test module: test/suite/test_base01

Traceback (most recent call last):

  File "/usr/lib64/python3.6/unittest/loader.py", line 153, in loadTestsFromName

    module = __import__(module_name)

ModuleNotFoundError: No module named 'test/suite/test_base01'

----------------------------------------------------------------------

Ran 1 test in 0.000s

FAILED (errors=1)

I installed wiredtiger to /opt/wt-10.0 (configure command is: CFLAGS="-g -O0" ./configure --enable-snappy --enable-python --prefix=/opt/wt-10.0)

wiredtiger ((mongodb-4.4.1) *%)$ ls /opt/wt-10.0/lib

libwiredtiger-10.0.0.so  libwiredtiger.a  libwiredtiger.la  libwiredtiger_snappy.a  libwiredtiger_snappy.la  libwiredtiger_snappy.so  libwiredtiger.so  pkgconfig

The wt python library was installed to /usr/local/lib64/python3.6,

Anything wrong ?

[bai...@gmail.com]

If I do this, there's the following error: 

So how should I run a single py ?

[Luke Chen]

Can you try using only the test file name (instead of the full path) for the last argument

e.g. $ python3 ./test/suite/run.py test_base01.py

or even $ python3 ./test/suite/run.py test_base01

The testing framework will regex the input (value of the last argument) to figure out the right set of test files/classes. 

Also, the wiki page assumes you "make" under the "build_posix" directory and generate all library files under "build_posix/.libs" folder. 

In your case, it looks the library files were generated under /opt/wt-10.0/lib. So you may try prefixing env LD_LIBRARY_PATH="lib" for the python command if you run it under /opt/wt-10.0

To view this discussion on the web visit https://groups.google.com/d/msgid/wiredtiger-users/e0c979f6-669d-4d0d-9055-ff6099eec988n%40googlegroups.com.

[bai...@gmail.com]

Actually I tried, (I thought the libraries in .libs are same as in /opt/wt-10.0/lib, so I haven't uploaded the case), 

[Luke Chen]

This error is actually different from the initial error you reported. I'm not able to reproduce it locally. 

Can you confirm the OS variant/version, SWIG version, and Python version that you used in your environment? 

Don,

Any idea what could be possibly wrong that generated the argument TypeError? 

Luke

To view this discussion on the web visit https://groups.google.com/d/msgid/wiredtiger-users/a6cc197e-baf6-4dcb-80e1-ddbc19a49a8cn%40googlegroups.com.

[bai...@gmail.com]

OS is like redhat , kernel version 4.14

Python3 : 3.6.8

swig: 4.0.2

Is is becasue swig's version is too new ?

[bai...@gmail.com]

related issue :

https://jira.mongodb.org/browse/WT-6091

https://jira.mongodb.org/browse/WT-4815

They are still open.

[Donald Anderson]

Hello,

It looks like WT-6091 may be the problem here.  The workaround for WT-6091 is to remove lang/python/wiredtiger_wrap.c (from the top level of the tree, it is not relative to build_posix).  Then run make again.

Which version or branch of wiredtiger are you building?  I was not aware that this can still happen in the develop branch.

- Don

—

Don Anderson

d...@mongodb.com

To view this discussion on the web visit https://groups.google.com/d/msgid/wiredtiger-users/34881d18-3f61-45bc-8b2b-cb406d67980en%40googlegroups.com.

[bai...@gmail.com]

Wow, I tried this and worked !

I'm on the mongodb-4.4.1 tag, 579966149b9094922f22d9bd7e8f3b8b9db1ccf7

[bai...@gmail.com]

Thanks to Luke and Don.

Another thing, I compile Evergreen and see the output of `evergreen -h` (many commands). I want to run it on local machine, is that possible ? (I've noted that there're evergreen.yml under wiredtiger source tree.)

 I didn't see steps about how to make use of those commands from https://github.com/evergreen-ci/evergreen/wiki

[Luke Chen]

It's great to hear the workaround solved your issue. 

With regard to Evergreen, I guess you are attempting to create a patch build manually using the CLI. You will need an Evergreen credential to be able to use the CLI. 

The Evergreen infrastructure is maintained internally by the MongoDB team, and I doubt it currently opens access to community members. 

With that being said, Evergreen (and any CI system) just provides an automatic way of running tests. In most cases one should be able to run those tests manually without relying on Evergreen. If you have any specific need e.g. to access some Evergreen test failure log or repro a failure, I'm sure one of us could offer helps. 

Luke

To view this discussion on the web visit https://groups.google.com/d/msgid/wiredtiger-users/e682f227-e544-4d05-a679-5eb2cc6ea6den%40googlegroups.com.

[bai...@gmail.com]

Good to know. I see this article https://engineering.mongodb.com/post/evergreen-continuous-integration-why-we-reinvented-the-wheel , and I consider it to be a fantastic CI system ! So I'm kind of curious.