0.1.5 release check-in

[Kriti Godey]

Hey everyone,

We need to get the February release out soon. We don't have a lot of changes, so the QA should be quick and easy, hopefully. FYI, I archived our internal 0.1.4 Basecamp project and moved the remaining open items to a "Docs improvement" list in the 0.1.5 project.

Here are the some to dos and question, based on both what's in the milestone and other stuff we've been discussing, broken down by person:

• Anish
• Will the new demo infrastructure setup be done by next week in time for the release?
• What is the ETA on the following issues?
• New connection creation flow does not handle schema creation failure scenarios · Issue #3420 · mathesar-foundation/mathesar
• Attempting to add connection with both Library Management and Movie Collection schema results in a 502 · Issue #3423 · mathesar-foundation/mathesar
  
• Brent
• When can Demo mode results in 5XX · Issue #3433 be closed?
• Can you take responsibility for setting up any testing infrastructure we need for QA? I'm hopeful we won't need any.
• Can you also take responsibility for post-release deployments?
• Can you also be on standby to help with docs updates if there's any feedback from our review?
  
• Ghislaine
• You have testing installation assigned to you, can you please do that ASAP?
• What is the ETA on the QA testing scripts for release?
  
• Kriti
• I need to do a holistic docs review, mainly.
• I'm doing this organizational / project management work.
• I'll do the post-release marketing work as well.
  
• Sean
• Thanks for starting on the release notes! I'm assuming you'll keep release notes updated if more things get merged.
• I'll let you know when I've done my docs review so you can do yours, I'll be prioritizing it.
• Can you work on these two issues?
• Update "Connect to an external database server" docs · Issue #3428 · mathesar-foundation/mathesar
• UI points to missing user documentation when deleting a connection · Issue #3438 · mathesar-foundation/mathesar
• Can you take responsibility for the following lists in the Basecamp project?
• Release Notes
• Publication
• QA
• If we don't need to do much QA, that's fine, I'd just like you to make sure we have a plan and execute it, you can recruit Brent for infrastructure and additional testers if needed.
  
• Publish Release
• Post-Release Cleanup (except for organizing the retrospective, I can do that)
• There's also some tasks in the Process Improvements section that you added and I don't have context for, can you also take responsibility for those?
  
• Pavish
• Can you prioritize reviewing and merging Constrain the width of the connections page by seancolsen · Pull Request #3439 · mathesar-foundation/mathesar
• What is the status of whatever's blocking ordering translations, and can we prioritize that and order the translations soon?
  

Please respond here with answers, and thanks! Let me know also if I'm missing any release work.

Thanks!

Kriti

[Brent Moran]

• When can Demo mode results in 5XX · Issue #3433 be closed?
• 
  

• I have a branch that fixes the problems there. I'll raise a PR if we want, but the eventual plan is still to get rid of "Demo Mode" in the actual code and handle it on an infrastructure level.
  

• 
  
• Can you take responsibility for setting up any testing infrastructure we need for QA? I'm hopeful we won't need any.
• 
  

• Yes.
  

• 
  
• Can you also take responsibility for post-release deployments?
• 
  

• Yes.
  

• 
  
• Can you also be on standby to help with docs updates if there's any feedback from our review?
• 
  

• Yes.
  

[Brent Moran]

Wow. That was not what the formatting was showing in the editor.

[Ghislaine Guerin]

I will prioritize the installation testing for today and ETA for scripts is Friday. I've updated due dates on basecamp.

[Anish Umale]

Will the new demo infrastructure setup be done by next week in time for the release?

Yes, It should be up and running by the end of this week. 

What is the ETA on the following issues? 

• New connection creation flow does not handle schema creation failure scenarios · Issue #3420 · mathesar-foundation/mathesar 
• Attempting to add connection with both Library Management and Movie Collection schema results in a 502 · Issue #3423 · mathesar-foundation/mathesar

I'll finish PRs for both of these issues by Monday.

[Sean Colsen]

Kriti,

• Regarding Update “Connect to an external database server” docs:

  Sorry, but I don’t think I can take this task solo. I’ll need help from someone else with a better understanding of the underlying content. It looks like that page has mostly been written by Anish. He and I could potentially do some pairing on it, but given my scheduling challenges, I don’t think it’s worth trying to schedule that to happen before the v0.1.5 release. At the very least we need to remove references to MATHESAR_DATABASES. I wonder if Anish could handle that on is own. What do you think Anish?

  At a high level, I have some qualms with this page being hard to follow for a reader who doesn’t already understand the context. I would try to say more here (or in the ticket), but it would be hard via async communication. To express my concerns, I’ll need to ask many small and sequentially-dependent clarifying questions to someone with a better understanding of the docs goals behind this page.

• Regarding UI points to missing user documentation when deleting a connection

  I opened a PR, Temporarily hide link to missing docs page, which just removes the offending UI. I don’t have time to write the docs content before the release, so I added an issue, Improve user docs about Mathesar schemas, to track that work.

I have some other responses and questions to other points in your email but unfortunately don’t have time to write them right now. I’ll try to respond again tonight with more.

​

[Pavish Kumar Ramani Gopal]

> Can you prioritize reviewing and merging Constrain the width of the connections page by seancolsen

Yes, I can do that by the end of this week.

> What is the status of whatever's blocking ordering translations, and can we prioritize that and order the translations soon?

I've deprioritized it. I intended to take a look at this sometime next week whenever I get time in-between my shifting and the conference.

I can prioritize this but I don't think we'd be able to get the translations ready from the ordered service for 0.1.5, considering the time frame. Is it okay if I only pick this up next week?

Adding to the list of items I need to do for the release,

I will also debug and fix the regression that Ghislaine reported today: Exploration share link not working.

[Kriti Godey]

Okay, thanks everyone. We’ll cut the release branch on Monday or Tuesday, do QA, and release by Thursday.

Anish

Thanks for the update! We’ll wait to cut the release until your work on those issues is merged. Tuesday is our cutoff though, so if you can’t get those fixes merged in by Tuesday, we’ll release without them.

Also, you’ll be doing some QA next week, so please set aside some time for that.

Brent

You said:

I have a branch that fixes the problems there. I’ll raise a PR if we want, but the eventual plan is still to get rid of “Demo Mode” in the actual code and handle it on an infrastructure level.

Is there any point to this if Anish is getting the demo infrastructure done this week? I assume we won’t have enough time to delete the “demo mode” code for 0.1.5, even if Anish does get the infrastructure up and running. We can still deploy the demo with demo mode on the new infrastructure, right?

Also, please review / merge Anish’s work ASAP and be on standby on Mon / Tue to deploy the release to the internal server once Sean has cut the release branch. I also made a Basecamp task for you to help Ghislaine with the installation issue she faced.

Ghislaine

Thanks for testing installation, and good to know about the QA scripts. We don’t need the QA scripts for this release, but we can use them if they’re ready.

Sean

• I’ll take a look at the Update “Connect to an external database server” docs when I do my review, let’s go from there.
• I’ll also take a look at the other docs and your PR at the same time.
• Please also prioritize reviewing Pavish’s PR for the exploration share link issue if it’s ready

FYI to everyone else: Sean and I discussed the rest of his points on Matrix and we’re good to go.

Pavish

• Let’s not worry about the translations for 0.1.5, I don’t think you necessarily need to prioritize it next week if it won’t go into 0.1.5. We should still do it soon, but please prioritize your PGConf talk and the permissions issue.
• We’ll wait until Tuesday to cut the branch, if you’re able to get the exploration share link issue merged before then, that would be great, otherwise it needs to wait for 0.1.6.

---

Does anyone have any further questions or thoughts? Are everyone’s priorities clear?

Thanks!

[Kriti Godey]

Brent and Pavish, please prioritize ASAP review/merge of https://github.com/mathesar-foundation/mathesar/pull/3448 and https://github.com/mathesar-foundation/mathesar/pull/3454

[Kriti Godey]

Also meant to mention this in the previous email, but Sean, I assume you'll cut the release branch after those are merged and coordinate with Brent to deploy the release and start QA.

[Pavish Kumar Ramani Gopal]

Kriti & Sean, I reviewed https://github.com/mathesar-foundation/mathesar/pull/3454 and noticed that the root cause wasn't addressed. I took some time to debug it, and it turned out to be an issue with the `common_data` formation on the backend. I closed that PR, opened https://github.com/mathesar-foundation/mathesar/pull/3456, and set it to auto-merge since it was a small enough change. It's assigned to Brent to take a look post merge.

I think I don't have any pending task that's blocking the release, please let me know otherwise.

[Sean Colsen]

Thanks Pavish — looks good!

[Brent Moran]

I merged Anish's PR addressing the 502 when loading the movies collection. Assuming the slow loading was the root cause, that should solve the problem. However, that was a problem that went unnoticed until we were testing on the testing infrastructure on GCP, and we should double-check that once it's set up.

[Kriti Godey]

Thanks Brent! I also made this docs PR for you to review, plus I assume you're working on the demo mode fix. 

Sean, I assume since we don't need QA for the docs PR, we can merge this into the release branch after it's cut if needed (tomorrow or Wed).

[Kriti Godey]

Also, Anish, I haven't heard from you about New connection creation flow does not handle schema creation failure scenarios · Issue #3420 · mathesar-foundation/mathesar so I assume that won't make it to this release, I've moved it to 0.1.6.

Also, since you're at PGConf India this week, I'm assuming you won't have time for QA, please let me know otherwise. Ghislaine, you'll need to take over Anish's QA tasks as well, I reassigned them in Basecamp to you.

[Sean Colsen]

I’ve been working on cutting the release but I’m stuck and I need help.

First, I’ll give a few updates, because that will help explain where I’m at:

• As I’ve been proceeding with the release process, I’ve been updating our docs along the way. My goal with these updates is to gently nudge our process more towards automation by swapping prose for bash commands that the reader can copy-paste.

• I just pushed a pretty big commit to the wiki which re-works our release docs. There’s now a top-level Release process page with many different sub-pages. The release process is now broken up into just a small handful of independent steps — each with their own documented procedure.

• “Cutting a release” is the step I’m on now. You can see this page is all copy-paste commands now!

I’m stuck on the docker login command. (Or, as I run on my machine, sudo docker login.)

I’m using the credentials in 1Password. I’ve tried it several times, but I see this:

Error response from daemon: Get “https://registry-1.docker.io/v2/”: unauthorized: incorrect username or password

Any advice?

It’s worth noting that, prior to my docs changes, the old docs just said:

Ensure you’re logged into Dockerhub. (Credentials are in 1Password.)

I’m guessing this means I need to run docker login, but I’m not sure. (Although I helped with the previous release, Pavish still did this part, so the process is new to me.) As a troubleshooting step, I actually tried logging into https://hub.docker.com/ using our credentials but it said I can’t log in there using an organization account.

I’m turning in for the night, but I expect to be back at my desk around 13:00 UTC Tuesday.

​

[Brent Moran]

@Sean Colsen

You can't log directly into the org account. You have to login using your own docker credentials.

Regarding that,  I just double-checked the org setup and it seems you never accepted the invitation to join the Mathesar org on Docker Hub. I resent the invitation; it should land in whatever email you used to sign up for Docker Hub, and should contain a link to finish the sign up.

After following the link and finishing the sign up process to join the Mathesar org on Docker Hub, you should have rights to push to our Docker repos, after logging in at the CLI using your own credentials.

Brent

[Brent Moran]

Clearly, we also need to update that section of the wiki.

[Pavish Kumar Ramani Gopal]

Team,

I created a Docker cloud builder and used it to build and publish a test image to DockerHub and it worked great.

It only took a total of 6 minutes to build and publish both the amd and arm images of mathesar-prod!!!

Here are the instructions on how to do this. We can update the release process docs if we are all in agreement in using this moving forward (I'm not entirely sure about the payment stuff, but it looks like we get 400 minutes monthly which should be more than enough).

1. Update your docker engine & buildx versions.

• My previous buildx version did not contain the cloud driver. Here's my current version:
• Docker engine:  25.0.3
• Buildx (docker buildx version)

• github.com/docker/buildx v0.12.1-desktop.4 6996841df2f61988c2794d84d33205368f96c317 

• I had Docker desktop app installed and was able to update the versions via its UI. You might have to check for instructions on how to do this for your OS/environment.
  

2. Login to docker in your local environment using `docker login`.

3. Create a local instance of our cloud driver:

docker buildx create --driver cloud mathesar/release-builder

 

4.  Use the cloud builder to build and publish our images:

docker buildx build \
  -t mathesar/mathesar-prod:0.1.5 \
  --builder=cloud-mathesar-release-builder \
  --platform=linux/amd64,linux/arm64 \
  --push \
  --build-arg PYTHON_REQUIREMENTS=requirements-prod.txt .

• The above command will build our prod setup for amd and arm using our cloud builder, tag it as 0.1.5, and publish it to DockerHub.
• The only change from the command present in our release docs is the value of `--builder=`

5. Logout of Docker: `docker logout`.

@Sean Colsen If you'd like to give this a go and publish the release images for 0.1.5, the above instructions should be helpful.

[Kriti Godey]

Nice work, Pavish! As mentioned on the release check in meeting, we have 400 free minutes a month so no need to worry about payment yet.

[Kriti Godey]

Release is out, nice work everyone.