UI testing meeting minutes

[Antonin Delpeuch (lists)]

Hi all,

Here is a summary of what was discussed in the UI testing meeting
(attended by Florian, Thad, Tom and me).

Florian demonstrated how the Cypress tests can be run. It requires
running an instance of OpenRefine in the background.

We discussed the directory structure for UI tests: how closely should it
match the structure of the code? Since those are integration tests they
do not actually need to be closely tied to the code, but it would be
nice to follow a similar architecture for the sake of uniformity.

Tom brought up the question of whether these new tests could catch a bug
like the one we introduced in the scatterplot facet during the jQuery
migration. It seems that by default the tests would not catch this sort
of rendering issue - we would need pixel-based comparison. There are
tools to do that in Cypress, we need to evaluate how much resources that
requires and if/when we want to enable it.

We discussed the integration in Travis. We could have one Travis job per
browser.

Florian demonstrated how we can inspect the tests in cypress.io (with a
video of the test run).

We discussed how to integrate the work done so far in the codebase: just
do a PR to master.

Documentation of the testing framework will be added in the
Docusaurus-based documentation, in the developer section.

Best,

Antonin

[Thad Guidry]

Great minutes! Well done Antonin and Florian!

Thad

https://www.linkedin.com/in/thadguidry/

--
You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/76439469-531d-75d7-3803-91e625180f5b%40antonin.delpeuch.eu.

[Florian Giroud]

Hello, here are the minutes from the meeting of Thursday 11 December 2020
(attended by Thad, Kush and me).

The discussion was mainly a follow-up about this thread : https://groups.google.com/g/openrefine-dev/c/cUytDLYSq3w

All questions answered, that was great meeting, thanks all !

How to organise the test suite ?

Basically we agreed on following the visual organisation of OpenRefine to organise the tests files and folders.

It's also a good idea to stick on how it's named in the codebase when we don't know how to name an element.

Then the newcomers can understands the test suite easily

What to test, and how to prioritize?

Instead of what's written in the thread (do the quick wins first), it's best to focus on the parts of OpenRefine that 

• are subject to change
• had regressions
• constitutes the core features of OpenRefine
• are mentioned in the videos/tutorials, because that's what's everyone tries first

The following parts are identified as a priority

 - Edit-cells and all it's children (in the columns menu)

 - Facets

 - Scatterplot

The following parts are at the bottom of the list

- Reconcile

- Transpose

- Clustering

Those are important blocks but need a lot of inputs from developers to first understand how it works, then understand what makes a good test for them.

Placeholders tickets will be created for those, where the community can take the time to discuss how to approach them.

Then, for all components, we should have

- at least one MVP test to check the core features of the component, without deep diving into all the possibilities

- X tests for all the options, according to the priority

My next step is to create tickets for everything, in order to have an implementation roadmap early January.

Also: Thad will take over on the firefox issues

Thank you
- Florian

[Antonin Delpeuch (lists)]

Thanks a lot all three and sorry I couldn't attend, I was presenting
OpenRefine with Allana in an event run by CZI, our current funder.

Antonin

On 11/12/2020 08:53, Florian Giroud wrote:
> Hello, here are the minutes from the meeting of Thursday 11 December 2020
> (attended by Thad, Kush and me).
>
> The discussion was mainly a follow-up about this thread :
> https://groups.google.com/g/openrefine-dev/c/cUytDLYSq3w
> All questions answered, that was great meeting, thanks all !
>

> *How to organise the test suite ?*

> Basically we agreed on following the visual organisation of OpenRefine
> to organise the tests files and folders.
> It's also a good idea to stick on how it's named in the codebase when we
> don't know how to name an element.
> Then the newcomers can understands the test suite easily
>
>
> What to test, and how to prioritize?
> Instead of what's written in the thread (do the quick wins first), it's
> best to focus on the parts of OpenRefine that 
>

> * are subject to change
> * had regressions
> * constitutes the core features of OpenRefine
> * are mentioned in the videos/tutorials, because that's what's

> <http://cypress.io> (with a

> video of the test run).
>
> We discussed how to integrate the work done so far in the
> codebase: just
> do a PR to master.
>
> Documentation of the testing framework will be added in the
> Docusaurus-based documentation, in the developer section.
>
> Best,
>
> Antonin
>
> --
> You received this message because you are subscribed to the
> Google Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to openrefine-de...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/openrefine-dev/76439469-531d-75d7-3803-91e625180f5b%40antonin.delpeuch.eu.
>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com

> <mailto:openrefine-de...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openrefine-dev/12cc3c85-ac02-45f7-bfa7-71d7b1dd3d03n%40googlegroups.com
> <https://groups.google.com/d/msgid/openrefine-dev/12cc3c85-ac02-45f7-bfa7-71d7b1dd3d03n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Florian Giroud]

Here are the minutes from the meeting of 18 February
Participants: Antonin Delpeuch, Florian Giroud

Topics discussed

1. Tests stability
Stability have been greatly improved over the last couple of weeks. Few runs remains flaky though, to be investigated and re-runned case-by-case
Overal recommendation is to re-run github actions for failed builds for now

2. Split&Join, 
All questions answered, looks like there are no bugs, 
https://github.com/OpenRefine/OpenRefine/issues/3423
I will update the tests

3. Wikidata extension
Reconciliation should be implemented first, even before the wikidata extension

Thanks

[Florian Giroud]

Here are the minutes from the meeting of 15 April
Thad, Antonin, Florian

A. Parallelism:
We feel the need to run tests in parallel, but we first want to be sure we are using as much as we can from the existing VM
We have a working PoC at https://github.com/fgiroud/OpenRefine/tree/cypress-parralel, not meant to be merged because of a dirty git history

The drawbacks are the following

• there is a dependency on Cypress dashboard (him being the load-balancer for the tests)
• users, therefore, need a cypress account, and a repo configured with a secret and a key
• we are not sure we are using exisring VMs at their maximum capacity

A couple of directions to be investigated

• unofficial plugins that would allow tests to run in parallel on a single VM (https://github.com/tnicola/cypress-parallel)
• find a way to measure the memory and CPU consumption of Github VMs

Thad to investigate memory/CPU measurement, Florian to try to implement a solution with parallelism on a single VM

B. Next tests:
Clustering is a good candidate for the next tests
Reconciliation is almost finished, to be submitted very soon

- Florian

[Thad Guidry]

I've added a few updates to my comments in issue:  https://github.com/OpenRefine/OpenRefine/issues/3817#issuecomment-820427744

Thad

https://www.linkedin.com/in/thadguidry/

https://calendly.com/thadguidry/

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/df611436-ae72-4966-b41a-7c87f10ee065n%40googlegroups.com.

[Antoine Beaubien]

Hi Florian,

   I'm trying to run the UI Tests, but I get errors because it have some preferences set. Can the tests be launch in a separate Workspace?

Regards,

   Antoine

[Florian Giroud]

Hi Antoine, OpenRefine needs to be running but the tests are not particularly tied to it.
What you can do is to start OpenRefine with a separate directory, ./refine -x refine.data_dir=/tmp/ui_test
And you can replace /tmp/ui_test by a path to a folder on your OS

Best,
- Florian

[Antoine Beaubien]

Hi Florian,

   Thanks for the answer. I realized I didn't asked correctly my idea.

   Seeing that testing the UI can alter someone workspace, and not work because of previous preferences of the user, could we have a flag for the UI tests? Something like ./refine test_ui?

Regards,

   Antoine