CD/CI and Functional tests; random failures

223 views
Skip to first unread message

Antoine Beaubien

unread,
Apr 13, 2022, 6:02:21 PM4/13/22
to OpenRefine Development
Hi,

   Quite a few times in the last weeks, I witnesed that the validation checks are failing in the functional tests, for thing it shouldn't.

   Specifically, I see that error often:
 CD:CI failures.jpg

AssertionError: Timed out retrying after 4000ms: Expected to find element: `body[ajax_in_progress="true"]`, but never found it.

1) can we put a higher timeout? 5 s. ?
2) can we code into OR JS objects events that would keep Cypress informed that OR is working, instead of having Cypress pull for status every X s.?
3) or something else?

Most of the time, asking to re-run failed jobs is enough. But we got to see it and do it, so it's not ideal.

Regards,
   Antoine

Antoine Beaubien

unread,
Apr 24, 2022, 8:53:33 PM4/24/22
to OpenRefine Development
Also this:

Screenshot 2022-04-24 20.51.19.png

Antoine Beaubien

unread,
Apr 24, 2022, 9:18:45 PM4/24/22
to OpenRefine Development
I have a question for you, Antonin:

Here, I've ask for a re-run of all tests. Is it possible that our cypress test suite doesn't handle re-run well?

Regards,
   Antoine


Screenshot 2022-04-24 21.16.39.png



Antonin Delpeuch (lists)

unread,
Apr 25, 2022, 1:29:17 AM4/25/22
to openref...@googlegroups.com

Hi Antoine,

Yes I have also noticed that when running the entire test suite sequentially, tests often fail.

Florian has told me that we are not actually supposed to run the entire test suite sequentially but rather run individual tests.

Antonin

--
You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/c927b901-58b9-480b-8a3b-f027e9a2db4cn%40googlegroups.com.

Antoine Beaubien

unread,
Apr 25, 2022, 3:04:36 AM4/25/22
to OpenRefine Development
Thanks for the answer.

The thing is, GitHub doesn't alway offer other options than re-runnning all the tasks.

Re-run all tests.jpg

Another question: will the « re-run failed jobs » will succeed?
(Obviouly not if the tests fails in itself, but in the case of timeout from Cypress… which are common.)

Re-run all tests2.jpg

Regards,
   Antoine
Reply all
Reply to author
Forward
0 new messages