Running TUF tests

[Kainaat Singh]

Hello,

I am a beginner in open source contribution and looking to contribute to this project via issue Remove uses of from __future__ import #1297.

I have removed all the occurrences but I would like to run the tests in the tests folder. Currently, I am trying to run these tests in the develop branch. But I get the same error on each test:

tests.utils.TestServerProcessError: '/home/tuf/tests/simple_server.py did not print port message as first stdout line as expected!'

Any help would be appreciated! Thanks!

[Santiago Torres]

Hi, Kainaat nice to meet you!

So, how are you running the tests? My two guesses would be:

1. A server is already running on the test port so the server
crashes/hangs somehow.

2. A server is really just not running for some reason.

Can you verify the test server is running? I'm always happy to peek at
a full log and help you parse it.

Cheers!
-Santiago

On Thu, Apr 01, 2021 at 11:06:51AM -0700, Kainaat Singh wrote:
> Hello,
>
> I am a beginner in open source contribution and looking to contribute to
> this project via issue Remove uses of from __future__ import #1297

> <https://github.com/theupdateframework/tuf/issues/1297>.

>
> I have removed all the occurrences but I would like to run the tests in the
> tests folder. Currently, I am trying to run these tests in the develop
> branch. But I get the same error on each test:
>

> *tests.utils.TestServerProcessError: '/home/tuf/tests/simple_server.py did
> not print port message as first stdout line as expected!'*

>
> Any help would be appreciated! Thanks!
>

> --
> You received this message because you are subscribed to the Google Groups "The Update Framework (TUF)" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to theupdateframew...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/theupdateframework/149919f7-f1cd-48a7-ad86-84f3d74a7bdcn%40googlegroups.com.

[Kainaat Singh]

Hi Santiago,

Thanks for your reply! Nice to meet you as well!

1. The first time I ran the tests without starting any server and I got the following result for every test:

.........E....EEEEEEEEEEEEE...............EE...............EEEE.....................................................................................................EE.EEEEEEE.EEE

======================================================================

ERROR: setUpClass (test_arbitrary_package_attack.TestArbitraryPackageAttack)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/home/kainaat/OpenSourceContributions/tuf/tests/test_arbitrary_package_attack.py", line 71, in setUpClass

    cls.server_process_handler = utils.TestServerProcess(log=logger)

  File "/home/kainaat/OpenSourceContributions/tuf/tests/utils.py", line 165, in __init__

    raise e

  File "/home/kainaat/OpenSourceContributions/tuf/tests/utils.py", line 160, in __init__

    self._start_server(timeout, extra_cmd_args, popen_cwd)

  File "/home/kainaat/OpenSourceContributions/tuf/tests/utils.py", line 179, in _start_server

    self._wait_for_port(timeout)

  File "/home/kainaat/OpenSourceContributions/tuf/tests/utils.py", line 256, in _wait_for_port

    raise TestServerProcessError(self.server + ' did not print port ' \

tests.utils.TestServerProcessError: 'simple_server.py did not print port message as first stdout line as expected!'

2. I followed the Quickstart guide and was able to run it successfully. Then to run the tests under /tests folder I setup the server in "tests" using "python3 -m http.server 8001" and ran the tests from the "tests" folder.

I get the same error as above for every test!  Do I need to create a tufrepo in the tests folder? 

Let me know if I am approaching this incorrectly and can share logs.

Regards,

Kainaat 

[Joshua Lock]

Hi Kainaat,

Thanks for your interest in tuf and for working on a patch for issue #1297!

We have some documentation on running the tests:

https://github.com/theupdateframework/tuf/blob/develop/docs/CONTRIBUTORS.rst#testing

As mentioned there, we also have a Tox setup to enable running the tests in a clean virtual environment. We use Tox to run the tests in our GitHub Actions based CI.

It should not be necessary to manually start any servers or create/copy any files, the tests are written to take care of these things. If running the tests as described in the CONTRIBUTORS document produces errors, that may be a bug in the tests. We can help resolve that if you provide some more information such as the environment (OS and Python version) you are running the tests on and full logs.

A GitHub Gist or Pastebin are a good way to share logs. We also have a #tuf slack channel on the CNCF Slack Workspace if you would prefer to discuss there.

Thanks,

Joshua

-- 
Joshua Lock
VMware Open Source Technology Center

To view this discussion on the web visit https://groups.google.com/d/msgid/theupdateframework/a81d1bc9-c719-45d3-bb28-6a90b5b5f9a8n%40googlegroups.com.