Reg: Documentation for contributors and testing improvements
26 views
Skip to first unread message
kusht...@gmail.com
Aug 30, 2020, 10:45:26 AM8/30/20
to OpenRefine Development
Hi Team,
As the call for proposals is open, I would like to present my interest in handling a few of the topics mentioned in the notice.
1.) Regarding documentation for contributors
For the following, I propose that we should add a header navigation item in our ongoing developing documentation site which may rather redirect to another page or a new website depending on what community feels appropriate. As a contributor when I first started contributing to OpenRefine, I felt there are various things which we can document more appropriately to have smoother contributor on-boarding.
- A welcome bot in our Gitter channel which will give out an automated response whenever a new contributor jumps in & say `Hello World`, we can change the world according to our need. In the automated response, we can redirect to the documentation site where all the required info will be populated.
- We need to document the architecture of OpenRefine & give a repository overview simplifying how our frontend code is organised and how our backend service works. OpenRefine's UI directory and structure need to be presented in a simpler manner by which the contributor can understand which piece of code is responsible for what, and which file should I change to see changes on the following page. Adding more to this, a video tutorial/walkthrough of OpenRefine's codebase will be cream to the top.
- Pull request process needs to be jotted down & the contribution workflow starting from how to get yourself an issue to how to get your PR successfully merged need s to be exposed.
- Other areas which can be separated from the basic contributing documentation can include documenting the release process, setting up IDE, and testing guidelines.
- One thing which I would like to propose from my side to establish governance in OpenRefine Organisation which would clearly highlight what roles we have like Org Member, Contributor, Maintainer and Admin.
2.) UI testing improvements
- Given a previous thread on UI Testing, the team have had enough discussion around why we need UI Testing & the tools which we may use. Out of the discussion Selenium, Cypress, and playwright were the more focussed ones with more bending towards playwright. So I will be making my further points ahead with the assumption that the team has decided to move on ahead with the playwright.
- As of now, OpenRefine's UI and Server are coupled together which means whether you want to spin a server or just UI, the later will spin up irrespective of your need. Here comes the challenging part, since the UI tests will be invoked through the continuous integration runner we may need to decouple the server and UI so that UI testing can be accomplished. Most of the testing tools are node-based which means you will have to use npm or yarn to invoke the testing scripts, run tests, and have successful assertions. With the present architecture, it would be difficult to test the UI, as you need a standalone UI which may serve as a demo for e2e tests & will make requests to the server either spin up in another CI job or hosted on Heroku or any other platform. The decoupling will not only be helpful for testing but for the future when the OpenRefine Team may want to move away from jQuery to a modular javascript framework.
- Having integration tests for functionalities like uploading datasets, sorting & filtering, scrolling & pagination, switch between the type of data-sets, rendering of menu items and rendering of dialogues to be taken care first before moving to complex functionalities which may take time.
- Having integration tests for reconciliation, reconciliation viewer, filter & sorting windows comes next as these actions put a load on the server and take several seconds to complete.
Apart from the above, I would like to propose a revamped website for OpenRefine. As there is on-going development for documentation website which looks quite modular, I suggest we should also consider revamping the openrefine.org to a modern fresh look. We can either switch the Jekyll theme or can migrate to gatsby for modernisation with a headless CMS to manage the blog posts too.
This thread serves as an initiation of the discussion around my proposed idea/suggestions. All of the above points need further discussion before finalising any.
All the suggestions/queries/changes are welcome from the OpenRefine team and community, all the above approach can be changed according to what community suggests.
Thanks & Regards
Kush Trivedi
Thad Guidry
Aug 30, 2020, 11:56:42 AM8/30/20
to openref...@googlegroups.com
Hi Kush!
First off, I'm going to mostly just be a listener on this thread but I did want to point out a few things where I am not advocating for anything in particular, but only sharing a few facts that you may or maynot be aware of already.
Regarding documentation for contributors,
1. As an aside of something to also look at; there's a great guide on "recommended community standards" which is all maintained by GitHub themselves here: https://opensource.guide/
2. And we haven't had to enable this repo setting yet "reported content in repository", but it is something to consider mentioning perhaps within any contributor docs.
https://docs.github.com/en/github/building-a-strong-community/managing-reported-content-in-your-organizations-repository
3. We also have a newly formed Code of Conduct committee and are also trying to improve our Governance docs.
Regarding the website,
We have talked about updating the website in the past but this was viewed as a lower priority for the team. We even talked about just using Docusarus itself as well for our small website. Its own docs mention:
Beyond that, Docusaurus 2 is a performant static site generator and can be used to create common content-driven websites (e.g. Documentation, Blogs, Product Landing and Marketing Pages, etc) extremely quickly
Many of Facebook's new open source websites are using Docusaurus 2 now.
While our main focus will still be helping you get your documentations right and well, it is possible to build this any kind of website using Docusaurus 2 as it is just a React application. Docusaurus can now be used to build any website, not just documentation websites.
Docusaurus has comparisons to Gatsby, GitBook, Jekyll, etc. Take a look at their main page: https://v2.docusaurus.io/docs/
We would be interested to hear your views on limitations within Docusaurus or other fameworks for features like Blog posting, etc., in particular, it would be nice to see a list of features that we would like or need to be supported in any chosen framework for our website to build some talking points (We might still have a issue or some comments from Martin or Owen on that already, so look around). One of the reasons we chose Docusaurus was the fairly low barrier to learning and working with it, its forthcoming i18n support, site search, as well as the future possibility for customization and expansion by using MDX and have JSX and React components embedded in the markdown.
Our community is expanding! And our small team is now growing with wonderful contributors such as yourself who want to help in many areas. We love that and look forward to more on this thread from you.
I like your enthusiasm Kush!
Kush Trivedi
Aug 30, 2020, 12:10:28 PM8/30/20
to openref...@googlegroups.com
Hi Thad,
Thanks for jumping on this!
I don't guess there are much limitation with Docusaurus, indeed it is great. But I would like to add few areas where gatsby might have am advantage over docusauraus :
1.) Gatsby is the framework with best SEO support and optimisation.
2.) Gatsby has lot of plugins and themes, which may be indeed quite useful is designing the homepage.
3.) As the docusaurus documentation itself mentions that :
Docusaurus tries to do one thing super well - be the best tool for writing and publishing content.
Gatsby may have an slight advantage in layout customization and UX which may want to provide.
These are just suggestions, I would have no issues if everyone agrees with Docusaurus. We can indeed make changes according to our need on the layouts.
Thanks and Regards
Kush
--
You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/CAChbWaMn9zvrrwUiJ0c4w8Ho4_RY5yeM7CtEPDFyEk%2BV3oDJQA%40mail.gmail.com.
Antonin Delpeuch (lists)
Aug 30, 2020, 12:17:54 PM8/30/20
to openref...@googlegroups.com
Hi Kush,
It's great to see many interesting ideas. I think it would be good to
focus on a much narrower set of improvements: each bullet point could be
discussed in its own thread, I would say. In this form, it will be hard
to keep the discussion focused.
Antonin
On 30/08/2020 16:45, kusht...@gmail.com wrote:
> Hi Team,
>
> As the call for proposals is open, I would like to present my interest
> in handling a few of the topics mentioned in the notice.
>
> 1.) *Regarding documentation for contributors*
> *
> *
>
> * A welcome bot in our Gitter channel which will give out an automated
>
> * Given a previous thread on UI Testing, the team have had enough
> <https://groups.google.com/d/msgid/openrefine-dev/44983640-3511-4dd8-9a41-539ca2723efcn%40googlegroups.com?utm_medium=email&utm_source=footer>.
It's great to see many interesting ideas. I think it would be good to
focus on a much narrower set of improvements: each bullet point could be
discussed in its own thread, I would say. In this form, it will be hard
to keep the discussion focused.
Antonin
On 30/08/2020 16:45, kusht...@gmail.com wrote:
> Hi Team,
>
> As the call for proposals is open, I would like to present my interest
> in handling a few of the topics mentioned in the notice.
>
> *
> *
> For the following, I propose that we should add a header navigation item
> in our ongoing developing documentation site which may rather redirect
> to another page or a new website depending on what community feels
> appropriate. As a contributor when I first started contributing to
> OpenRefine, I felt there are various things which we can document more
> appropriately to have smoother cont which means whether you want to spin a server or just UI, the later will spin up irrespective of your need. Here comes the challenging part, since the UI tests will be invoked through the continuous integration runner we may need to decouple the server and UI so that UI testing can be accomplished. Most of the testing tools are node-based which means you will have to use npm or yarn to invoke the testing scripts, run tests, and have successful assertions. With the present architecture, it would be difficult to test the UI, as you need a standalone UI which may serve as a demo for e2e tests & will make requests to the server either spin up in another CI job or hosted on Heroku or any other platform. The decoupling will not only be helpful for testing but for the future when the OpenRefine Team may want to move away from jQuery to a modular javascript framework.
> in our ongoing developing documentation site which may rather redirect
> to another page or a new website depending on what community feels
> appropriate. As a contributor when I first started contributing to
> OpenRefine, I felt there are various things which we can document more
Having integration tests for functionalities like uploading datasets,
sorting & filtering, scrolling & pagination, switch between the type of
data-sets, rendering of menu items and rendering of dialogues to be
taken care first before moving to complex functionalitiributor on-boarding.
sorting & filtering, scrolling & pagination, switch between the type of
data-sets, rendering of menu items and rendering of dialogues to be
>
> * A welcome bot in our Gitter channel which will give out an automated
> response whenever a new contributor jumps in & say `Hello World`, we
> can change the world according to our need. In the automated
> response, we can redirect to the documentation site where all the
> required info will be populated.
> * We need to document the architecture of OpenRefine & give a
> can change the world according to our need. In the automated
> response, we can redirect to the documentation site where all the
> required info will be populated.
> repository overview simplifying how our frontend code is organised
> and how our backend service works. OpenRefine's UI directory and
> structure need to be presented in a simpler manner by which the
> contributor can understand which piece of code is responsible for
> what, and which file should I change to see changes on the following
> page. Adding more to this, a video tutorial/walkthrough of which means whether you want to spin a server or just UI, the later will spin up irrespective of your need. Here comes the challenging part, since the UI tests will be invoked through the continuous integration runner we may need to decouple the server and UI so that UI testing can be accomplished. Most of the testing tools are node-based which means you will have to use npm or yarn to invoke the testing scripts, run tests, and have successful assertions. With the present architecture, it would be difficult to test the UI, as you need a standalone UI which may serve as a demo for e2e tests & will make requests to the server either spin up in another CI job or hosted on Heroku or any other platform. The decoupling will not only be helpful for testing but for the future when the OpenRefine Team may want to move away from jQuery to a modular javascript framework.
> and how our backend service works. OpenRefine's UI directory and
> structure need to be presented in a simpler manner by which the
> contributor can understand which piece of code is responsible for
> what, and which file should I change to see changes on the following
Having integration tests for functionalities like uploading datasets,
sorting & filtering, scrolling & pagination, switch between the type of
data-sets, rendering of menu items and rendering of dialogues to be
taken care first before moving to complex functionaliti:
sorting & filtering, scrolling & pagination, switch between the type of
data-sets, rendering of menu items and rendering of dialogues to be
> OpenRefine's codebase will be cream to the top.
> * Pull request process needs to be jotted down & the contribution
> workflow starting from how to get yourself an issue to how to get
> your PR successfully merged need s to be exposed.
> * Other areas which can be separated from the basic contributing
> your PR successfully merged need s to be exposed.
> documentation can include documenting the release process, setting
> up IDE, and testing guidelines.
> * One thing which I would like to propose from my side to establish
> up IDE, and testing guidelines.
> governance in OpenRefine Organisation which would clearly highlight
> what roles we have like Org Member, Contributor, Maintainer and Admin.
>
> 2.) *UI t**esting improvements*
> what roles we have like Org Member, Contributor, Maintainer and Admin.
>
>
> * Given a previous thread on UI Testing, the team have had enough
> discussion around why we need UI Testing & the tools which we may
> use. Out of the discussion Selenium, Cypress, and playwright were
> the more focussed ones with more bending towards playwright. So I
> will be making my further points ahead with the assumption that the
> team has decided to move on ahead with the playwright.
> * As of now, OpenRefine's UI and Server are coupled together which
> use. Out of the discussion Selenium, Cypress, and playwright were
> the more focussed ones with more bending towards playwright. So I
> will be making my further points ahead with the assumption that the
> team has decided to move on ahead with the playwright.
> means whether you want to spin a server or just UI, the later will
> spin up irrespective of your need. Here comes the challenging part,
> since the UI tests will be invoked through the continuous
> integration runner we may need to decouple the server and UI so that
> UI testing can be accomplished. Most of the testing tools are
> node-based which means you will have to use npm or yarn to invoke
> the testing scripts, run tests, and have successful assertions. With
> the present architecture, it would be difficult to test the UI, as
> you need a standalone UI which may serve as a demo for e2e tests &
> will make requests to the server either spin up in another CI job or
> hosted on Heroku or any other platform. The decoupling will not only
> be helpful for testing but for the future when the OpenRefine Team
> may want to move away from jQuery to a modular javascript framework.
> * Having integration tests for functionalities like uploading
> spin up irrespective of your need. Here comes the challenging part,
> since the UI tests will be invoked through the continuous
> integration runner we may need to decouple the server and UI so that
> UI testing can be accomplished. Most of the testing tools are
> node-based which means you will have to use npm or yarn to invoke
> the testing scripts, run tests, and have successful assertions. With
> the present architecture, it would be difficult to test the UI, as
> you need a standalone UI which may serve as a demo for e2e tests &
> will make requests to the server either spin up in another CI job or
> hosted on Heroku or any other platform. The decoupling will not only
> be helpful for testing but for the future when the OpenRefine Team
> may want to move away from jQuery to a modular javascript framework.
> datasets, sorting & filtering, scrolling & pagination, switch
> between the type of data-sets, rendering of menu items and rendering
> of dialogues to be taken care first before moving to complex
> functionalities which may take time.
> * Having integration tests for reconciliation, reconciliation viewer,
> between the type of data-sets, rendering of menu items and rendering
> of dialogues to be taken care first before moving to complex
> functionalities which may take time.
> filter & sorting windows comes next as these actions put a load on
> the server and take several seconds to complete.
>
> Apart from the above, I would like to propose a revamped website for
> OpenRefine. As there is on-going development for documentation website
> which looks quite modular, I suggest we should also consider revamping
> the openrefine.org to a modern fresh look. We can either switch the
> Jekyll theme or can migrate to gatsby for modernisation with a headless
> CMS to manage the blog posts too.
>
> This thread serves as an initiation of the discussion around my proposed
> idea/suggestions. All of the above points need further discussion before
> finalising any.
>
> All the suggestions/queries/changes are welcome from the OpenRefine team
> and community, all the above approach can be changed according to what
> community suggests.
>
> Thanks & Regards
> Kush Trivedi
>
> the server and take several seconds to complete.
>
> Apart from the above, I would like to propose a revamped website for
> OpenRefine. As there is on-going development for documentation website
> which looks quite modular, I suggest we should also consider revamping
> the openrefine.org to a modern fresh look. We can either switch the
> Jekyll theme or can migrate to gatsby for modernisation with a headless
> CMS to manage the blog posts too.
>
> This thread serves as an initiation of the discussion around my proposed
> idea/suggestions. All of the above points need further discussion before
> finalising any.
>
> All the suggestions/queries/changes are welcome from the OpenRefine team
> and community, all the above approach can be changed according to what
> community suggests.
>
> Thanks & Regards
> Kush Trivedi
>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com
> <mailto:openrefine-de...@googlegroups.com>.
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/openrefine-dev/44983640-3511-4dd8-9a41-539ca2723efcn%40googlegroups.com
> <https://groups.google.com/d/msgid/openrefine-dev/44983640-3511-4dd8-9a41-539ca2723efcn%40googlegroups.com?utm_medium=email&utm_source=footer>.
kusht...@gmail.com
Sep 1, 2020, 12:51:25 PM9/1/20
to OpenRefine Development
Sure Antonin!
Anyone having any doubts/suggestions regarding the above-listed ideas and crude approach can reply to the thread & then we can start a new thread for each point :) .
Regards
Kush
Reply all
Reply to author
Forward
0 new messages