Release v0.1.3

[Rajat Vijay]

Team,

Here is the list of must-haves for this release: Milestone v0.1.3

Here is the list of good to-haves for this release: Next Release Milestone

Please go through the list and make sure

- Everything that you are working on in the current cycle is in the Milestone v0.1.3

- Anything that you think should be a must-have for this release and is not in Milestone v0.1.3, please reply with those issues links here for approval.

- If there is anything that does not make sense in Milestone v0.1.3, please reply in this thread.

Current State (2023-07-18)

- There are 43 open issues in Milestone v0.1.3

- There are 11 open issues in Next Release Milestone

Let me know if you have any questions/concerns.

Thanks

[Dominykas Mostauskis]

General question. Suppose I have issue A in the v0.1.3 milestone and a PR B that resolves issue A. Should PR B be in v0.1.3? Seems redundant to me, but maybe someone's workflow depends on this?

[Rajat Vijay]

Dom

- Yes, the PR should be in the same milestone as the issue that it fixes.

- I agree that it's redundant.

- I make sure to do this as part of my repo-admin responsibility for every open PR

- I can check for a GitHub action that automatically does this when I have some time during this cycle.

[Kriti Godey]

It's not redundant for me, it's helpful to track down which PRs are outstanding for a given release. I agree automation that syncs milestones between issues and related PRs would be helpful if available.

Rajat, I recommend going through all the issues in the two milestones you've listed and ensuring that they're all assigned and/or relevant to a project. That will help identify issues that may not belong, which you can then ask people about. I also recommend asking specific people to weigh in on specific issues (based on who created the issue), since asking everyone to take a look at every issue will be hard to track.

[Rajat Vijay]

Here is the summary of the items in the current milestone:

 Unassigned issues & not part of any project - Need confirmations from the tagged people if its intentional and to whom  these should be assigned to 

1. Provide a "Quicksave/Restore" function - @Dominykas Mostauskis 

2. Troubleshoot slow vertical scrolling of table rows - @Sean Colsen  / @Pavish Kumar Ramani Gopal (since its related to performance)

3. Improve GET /columns/ API performance - @Mukesh Murali 

4. Support Multiple Database Servers, supply db and creds via params - @Kriti Godey its related to SSO

5. Make it easy to install Mathesar when not using install.sh - @Mukesh Murali (related to installation improvements)

Assigned but not part of any project - Need confirmation from tagged people if they'll be able to do it before the release given they have their own separate projects

1. Type 1&2 upgrade container improvements - @Brent Moran 

2. Generate function list for whitelist-based dynamic defaults - @Dominykas Mostauskis 

3. Copying FK cells should copy the record summary instead of the PK value - @Sean Colsen 

4. Automate deploying internal Mathesar - This is assigned to @Brent Moran & @Dominykas Mostauskis who will be working on this?

I have assigned the following issues as per their respective projects - No action required here unless you think its a wrong assignment

1. helm chart would be great - @Mukesh Murali

2. Keep today as today in column default value - @Dominykas Mostauskis

3. Add hints/metadata for citext column type - @Dominykas Mostauskis

4. Incorporate field name into error messages when relevant - @Dominykas Mostauskis 

6. Add hints/metadata for point column type - @Dominykas Mostauskis

7. Demo users should only see their own database - @Dominykas Mostauskis

8. StatementError when using the Record Selector to search by a partially-entered UUID - @Dominykas Mostauskis 

9. Consider merging config into mathesar package - @Mukesh Murali 

10. Add support for SQLite as a additional datasource for the internal database - @Mukesh Murali 

11. Update the Mathesar superuser creation screen Django templates - @rajat vijay 

12. Generate zipapps - @Mukesh Murali 

13. Add a Docker image containing the Mathesar server along with a Postgres server - @Mukesh Murali 

14. Update documentation to include the Mathesar + Postgres docker image - @Mukesh Murali 

Others

1. Cannot import large csv tables - @Sean Colsen Can this be assigned to you since the problem explained by the user can be solved by your PR.

General Info

1. Updated the release date for the mileston to be 18th August.

2. Sending another email for the `Next Milestone`

[Kriti Godey]

I moved Support Multiple Database Servers, supply db and creds via params to the backlog.

[Sean Colsen]

I’ll reply to the items where I’m mentioned:

• Troubleshoot slow vertical scrolling of table rows – I’ve moved this to “Backlog”
• Copying FK cells should copy the record summary instead of the PK value – I’ve added this to my frontend fixes project
• Cannot import large csv tables – I’d like opinions from Dom and Kriti on this one. I think the most appropriate course of action would be to
  1. Change the milestone to “Backlog” or similar
  2. Leave it unassigned
  3. Comment on the ticket after we’ve made type inference optional, explaining that the upcoming release will offer a workaround but that we’re also still working on several other ways to improve inference and imports.
  4. Close the ticket if we get a response from the author indicating that they are satisfied with the workaround.
  5. Leave the ticket open if the author does not seem satisfied or does not reply.

​

[Kriti Godey]

I would keep Cannot import large csv tables in 0.1.3 and assign it to yourself, Sean. 

When type inference is made optional, we can then close it with a comment that says this fix will allow large CSV files to be imported as text (fixing the issue) but we’re also still working on several other ways to improve inference and imports. I don't think there's much benefit to leaving it open. Leaving it open implies further action will be taken on it, and I'm not sure what action that would be.

[Dominykas Mostauskis]

Provide a "Quicksave/Restore" function shouldn't be in the next release, and is not being worked on. The thread doesn't reflect this, but basically we've postponed working on backup/restore due to technical issues.

Automate deploying internal Mathesar shouldn't be associated with any release I think. @Brent Moran input?

[Brent Moran]

@Dominykas Mostauskis
You're correct. The internal deployment automation doesn't make sense
to be in the release. It's orthogonal.

Regarding https://github.com/centerofci/mathesar/issues/2647 , I've
moved it to the backlog. I think we should wait to see whether new
deployment methods even use the update server before spending effort
there.


On Thu, Jul 27, 2023 at 6:20 PM Dominykas Mostauskis

[Rajat Vijay]

Thanks for the update team.

There is still one pending item - Type suggestions broken. I have put this in the draft status since this is being discussed.

Current Status: There are 41 opened items

[Dominykas Mostauskis]

Concerning Type suggestions broken, I'd say that the type suggestions are not in fact broken, but rather when upgrading the dev environment there's an obscure bug that *sometimes* causes our Postgres library to silently not be updated. A lot of things are not clear to me about this issue, including impact. We do have a workaround for it (triggering reinstallation of our Postgres library). I'd say, let's not work on it this cycle, unless someone's discovered the exact cause. @Sean Colsen, @Brent Moran thoughts?

[Rajat Vijay]

Project level updates

Shareable Link (Pavish)

• Status: Pending Review
• Pending PRs

RSQLA1 (Brent)

• Status: Ready for Release

Installation Improvement 0.1.3 (Mukesh)

• Status: Pending
• Pending PR reviews
• Pending work - create a Debian Package for Mathesar

Frontend fixes (Sean)

• Status: Pending

i18n (Rajat)

• Status: Pending
• Pending PR reviews
• Pending documentation - can be done during the cool-down period
• Pending - ESLint rule catching non-translated strings - can be done during the cool-down period since will not lead to code-level changes

Backend Fixed (Dom)

Non-project related issues

1. Generate function list for whitelist-based dynamic defaults - Pending
2. Copying FK cells should copy the record summary instead of the PK value - Done

Next steps

Here is the next step that I propose:

Since we do not have a lot of things to release, let’s extend the cycle by 1 week. This way we most likely be able to move the following things to “Pending Release” -

1. Shareable Link
   1. Since only reviews are pending
2. Installation Improvement 0.1.3
   1. Except - Create debian or flatpak images for Mathesar
3. Either “Internationalization” or “Frontend fixes”
   1. Since “Frontend Fixes” is owned by Sean and he is also the reviewer for “Internationalization”
   2. We need to take a call on which one to prioritize

Thoughts?

[Rajat Vijay]

Sorry missed updates about the backend fixes project: 

Backend Fixed (Dom)

- Status: Pending 

[Kriti Godey]

We do not need to extend the cycle – we have a 2 week break between cycles. Let's focus on getting everything we can merged by this week and cut the release branch on Monday. That way, we can release next week, it'll still be during the break between cycles.

[Rajat Vijay]

Kriti, sounds good. 

There is one more question in the previous email: 

Should we prioritize reviewing i18n over Frontend fixes project or vice versa. 

Some data points to make this decision: 

1. Frontend fixed mainly consists of making column type inferences optional. This is requested by a user but there are some technical challenges to it as per Sean and we might not be able to land it in this week. 

2. i18n mainly requires review work by Sean and then fixing review comments by me. We might or might not be able to land it depending on the review feedback. 

[Kriti Godey]

We should prioritize i18n since it's a large / all encompassing feature and I think it's better to land it sooner rather than later. I assume Sean will continue work on making inference optional as time permits.

[Rajat Vijay]

Team,

I have cut a branch 0.1.3 & started the next steps mentioned in the Release process document.

Here is the issue for the release.

Here are the things that are still open & will not make it to the release

1. Installation Improvements 0.1.3

1. Cannot import large csv tables

1. 2023-07 Backend fixes

1. 1. Demo users should only see their own database

1. 1. Allow whitelisted dynamic defaults

1. 1. Generate function list for whitelist-based dynamic defaults

2. Make column type inference optional
3. i18n

Let me know if you have any questions.

[Kriti Godey]

Two questions:

• Are you working on smoke testing Mathesar to make sure there are no major regressions before release and that all functionality works? (Do we have a standard smoke test protocol?)
• Are you also double checking that all the work done this cycle is in a "releasable" state? (e.g. there's no half-done UX or documentation?)
• Since the whole "'make sure things don't get merged half-done" is something new we're doing this cycle, I think some manual work to double check is worth it.

Also, I'd like do a retrospective of the installation improvements project to figure out why no work got merged for the release. Mukesh, can you start an email thread about that?

[Kriti Godey]

Also, what's the plan with completing the i18n work and the installation work? My assumption is that i18n will get folded into ad-hoc frontend work for the next cycle, and installation work will be folded into next cycle's installation project, but I want to make sure we're all on the same page about that. If so, we should make sure the appropriate tickets are in the 0.1.4 milestone.

[Rajat Vijay]

Krit said:

Are you working on smoke testing Mathesar to make sure there are no major regressions before release and that all functionality works? (Do we have a standard smoke test protocol?)

Yes, I am doing the smoke testing as described in the "Testing" section of Release Process Wiki page

Are you also double checking that all the work done this cycle is in a "releasable" state? (e.g. there's no half-done UX or documentation?)

Yes, already checking that.

Also, what's the plan with completing the i18n work and the installation work? My assumption is that i18n will get folded into ad-hoc frontend work for the next cycle, and installation work will be folded into next cycle's installation project, but I want to 

make sure we're all on the same page about that.

The assumption is true for i18n. @Mukesh Murali can you comment for installation work?

If so, we should make sure the appropriate tickets are in the 0.1.4 milestone.

Got it. Will do today.

[Rajat Vijay]

Actually I just realized -

For the things that I mentioned will not be able to make it to the release -

Here are the things that are still open & will not make it to the release

1. Installation Improvements 0.1.3
2. Cannot import large csv tables
3. 2023-07 Backend fixes
   1. Demo users should only see their own database
   2. Allow whitelisted dynamic defaults
   3. Generate function list for whitelist-based dynamic defaults
4. Make column type inference optional
5. i18n

Can the respective owners please move it to the appropriate milestone. I have moved i18n open items to the v0.1.4 milestone.

[Dominykas Mostauskis]

I moved the mentioned issues, that belong to me, to the backlog milestone. Except, the main meta ticket for backend fixes for previous cycle, which is closed. I presume that a closed issue can be in whatever milestone.

Some of my relevant PRs are awaiting review. I could have asked the reviewer to prioritize them, but I didn't know when the cutoff for this release is. If that was communicated, I'm sorry I missed it. Maybe we can make the deadline more apparent next cycle. Maybe a reminder a week or two ahead of the deadline?

[Dominykas Mostauskis]

Btw, tangential, but I realised that the PRs I didn't get merged don't matter in a user deployment, so it's fine that they didn't make it. One is some missing tests that make sure we handle long identifiers well, and the other is for demo mode.

[Kriti Godey]

The demo mode does matter – we deploy the latest release to the demo server, so it means any problems with the demo don’t get resolved until the next release.

I didn’t know when the cutoff for this release is. If that was communicated, I’m sorry I missed it.

It was discussed earlier in this thread.

Maybe we can make the deadline more apparent next cycle. Maybe a reminder a week or two ahead of the deadline?

I think this is good feedback for whoever the release owner is next time.

[Dominykas Mostauskis]

> The demo mode does matter – we deploy the latest release to the demo server, so it means any problems with the demo don’t get resolved until the next release.

Any reason to not make an exception for a bug fix?

[Kriti Godey]

Any reason to not make an exception for a bug fix?

I’m not sure what you mean here, Dom.

---

Also general update: Brent is taking over releasing 0.1.3 with assistance from Pavish (since Rajat is unavailable due to a family emergency).

[Dominykas Mostauskis]

Any reason to not make an exception for a bug fix?

I’m not sure what you mean here, Dom.

I'm referring to fixing the db list in demo mode showing all databases (should only show the demo user's database). It didn't make it into the release, which, as I understand, is how we decide what makes it into the demo. I don't see a reason to not just deploy the `develop` branch to demo, so that this demo bug fix makes it into demo earlier than next release. Does this make sense?

[Kriti Godey]

I don’t see a reason to not just deploy the develop branch to demo

develop is unstable, e.g. see this critical regression reported today. Since the demo server is public-facing and is also used by the general public to evaluate Mathesar, we should not deploying potentially buggy code to it.

It’s fine to use develop for our internal server because only the core team accesses our internal server and it’s in our best interest to dogfood and find bugs early.

[Pavish Kumar Ramani Gopal]

Dom & Kriti,

> It didn't make it into the release, which, as I understand, is how we decide what makes it into the demo.

Since there is already a regression that needs a hotfix which would go into the release branch, we could potentially add this PR (Fix db list permissions in demo mode) additionally to the release as part of the hotfixes.

@Brent Moran would have to take a look and make a call on that.

Since Dom feels strongly that we get this fix out soon (as do I), this is a solution we could consider.

[Brent Moran]

Dear all,

There's a PR to fix the regression mentioned above by Kriti. Once that's merged, I'll redo the relevant steps of the release process. Depending on timing, I hope to make the release either tomorrow or Monday.

Regarding the db list permissions: I'm not actually good enough at Git to know how to get that change into the release branch cleanly. @Pavish Kumar Ramani Gopal Are you? If so, can you make that happen?

Brent

[Brent Moran]

Okay, after some back-and-forth with the team on this, I think we're going to have to wait till Monday to publish the release. Given that, we'll try to cherry-pick the commits associated with the PR to fix db list permissions in demo mode into the release branch via a fresh PR.

Pavish has agreed to do the git work on that, and I'll review.

Anish has agreed to review my PR fixing the column extraction fkey regression.

Brent

[Pavish Kumar Ramani Gopal]

Brent,

I raised a PR targeting the release branch with the commits from Dom's PR.

I did not have to cherry-pick commits since we had a clean merge commit and I merged that commit into the new branch, so we wouldn't face any possible merge related issues.

Pavish

[Kriti Godey]

Thanks Pavish and Brent, this sounds good to me. I assume we'll test the demo fix as part of manual testing of new features before we release.

[Brent Moran]

Hello all,

I went through the whole library demo using 0.1.3 today, checked other stuff ad-hoc, and didn't find any release-blocking problems. Thus, I've published the new images for the release. They're tagged on Dockerhub as 0.1.3. If anyone has time, please try those out. If I don't get any new issues or feedback indicating otherwise, my plan is to publish the release tomorrow.

Brent

[Brent Moran]

Okay!

Seeing no objections, I'm moving forward with publishing the release.

[Brent Moran]

Version 0.1.3 is officially released!

Here are the notes: https://github.com/centerofci/mathesar/releases/tag/0.1.3

A big thank you to everyone who contributed!

Brent