RE: Test subgroup meeting - Agenda 4/2/2025
Ying Xie
Folks,
On 4/2/2025, We have following topics to discuss, please attend:
- Policy of using conditional marker to skip tests.
We noticed that there had been a practice of filing github issue, skip test with the issue, and move one. This practice might be needed and useful to get PR test running for the short term. However, quite some issues had been open for very long time and start to hide legitimate issues that could cause regression to go into sonic repos.
As a community, we need to collectively define a game rule for using conditional markers.
- Issue triage.
Routine call for actions: Authors, please review follow test plan list, please reach out to me to schedule a presentation in test subgroup meetings when you are ready. I am trying to cleanup the backlog, but if you have presented the test plan in community. Please let me know.
Test plans can be presented: Pull requests · sonic-net/sonic-mgmt (github.com)
Thanks & Regards,
Ying
_._,_._,_
Links:
You receive all messages sent to this group.
View/Reply Online (#950) |
Reply to Sender |
Reply to Group |
Mute This Topic |
New Topic
Your Subscription |
Contact Group Owner |
Unsubscribe [ying...@microsoft.com]
_._,_._,_
Sunny Cai
Thank you for your message. I’m currently traveling and will have limited access to email from March 28 through April 6. During this time, my response may be delayed.
I’ll get back to you as soon as I can upon my return.
Best regards,
Sunny
On Mar 31, 2025, at 11:22 PM, 'Ying Xie' via sonicproject <sonicp...@googlegroups.com> wrote:
> Folks,
>
> On 4/2/2025, We have following topics to discuss, please attend:
>
> As a community, we need to collectively define a game rule for using conditional markers.
> Routine call for actions: Authors, please review follow test plan list, please reach out to me to schedule a presentation in test subgroup meetings when you are ready. I am trying to cleanup the backlog, but if you have presented the test plan in community. Please let me know.
>
> Test plans can be presented: Pull requests · sonic-net/sonic-mgmt (github.com)
>
> Thanks & Regards,
> Ying
>
> _._,_._,_
>
> You receive all messages sent to this group.
> View/Reply Online (#950) | Reply to Sender | Reply to Group | Mute This Topic | New Topic > Your Subscription | Contact Group Owner | Unsubscribe [ying...@microsoft.com]
> _._,_._,_
>
> You received this message because you are subscribed to the Google Groups "sonicproject" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sonicproject...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/sonicproject/DS7PR21MB3221C761FE4C2282FB3D705682AD2%40DS7PR21MB3221.namprd21.prod.outlook.com.
Ying Xie
Folks,
Thanks for joining today’s community meeting. We have a good discussion on the topic of test cases that are skipped by open issues.
Attached is the deck I shared today. In general, community reached initial agreement that we need to review / close the issues that are skipping test cases regularly and close them before we creating a new feature branch.
Follow up actions:
- Create a tag to mark issues that are used in conditional marker files to skip or xfail test cases. Here is the tag and query.
- Suggested to force an expiry from the conditional marker infrastructure. E.g. ignore issues that are opened for more than certain time (e.g. 6 months).
- Reviewers who reviews change in conditional marker files, please make sure that
- The referenced issue has ‘Impact test results’ tag set.
- The issue has a good justification why the test need to be skipped/xfailed. And timeline to address.
- Skip is allowed for feature gaps.
- Test failures/gaps should be using xfail.
- Investigated test failures that needs to stay for longer term should be updated to NOT using issue as condition.
- Community will review the “Impact test results” issues and close accordingly.
- @Xin Wang will organize another follow up discussion in the next east timezone test subgroup meeting.
- We will make the proposal to TSC accordingly. And start action in May when the new branch 202505 is created.
Thanks & regards,
Ying
From: soni...@lists.sonicfoundation.dev <soni...@lists.sonicfoundation.dev>
On Behalf Of Ying Xie via lists.sonicfoundation.dev
Sent: Monday, March 31, 2025 3:22 PM
To: Ying Xie <Ying...@microsoft.com>; vpunati <vpu...@google.com>; sonic-test-workgroup <sonic-test...@googlegroups.com>; sonicproject <sonicp...@googlegroups.com>; soni...@lists.sonicfoundation.dev;
朱升林(伍木)
<wumu...@alibaba-inc.com>; Ruan, Yixing <eddie...@alibaba-inc.com>; Srikishen Pondicherry Shanmugam <kish...@google.com>
Subject: [EXTERNAL] Re: [sonic-dev] Test subgroup meeting - Agenda 4/2/2025
Folks,
On 4/2/2025, We have following topics to discuss, please attend:
- Policy of using conditional marker to skip tests.
We noticed that there had been a practice of filing github issue, skip test with the issue, and move one. This practice might be needed and useful to get PR test running for the short term. However, quite some issues had been open for very long time and start to hide legitimate issues that could cause regression to go into sonic repos.
As a community, we need to collectively define a game rule for using conditional markers.
- Issue triage.
Routine call for actions: Authors, please review follow test plan list, please reach out to me to schedule a presentation in test subgroup meetings when you are ready. I am trying to cleanup the backlog, but if you have presented the test plan in community. Please let me know.
Test plans can be presented: Pull requests · sonic-net/sonic-mgmt (github.com)
Thanks & Regards,
Ying
_._,_._,_
Links:
You receive all messages sent to this group.
_._,_._,_