v0.5.0 Release thread

[Zack Krida]

Hi all,

The release branch  is cut for Mathesar v0.5.0. This week we will run QA and work on documentation for forms and other features included in the release.

Updates will be provided in this thread. After the QA process, we will publish the 0.5.0 release on Tuesday September 2nd. That's a day later than originally expected, to accommodate for the US federal holiday.

Please let me know if you have any questions or concerns.

Best,

Zack 

[Zack Krida]

Hi all,

Our QA tester and I have identified some bugs within forms, one of which seems critical to resolve. The first two are more minor and relate to disabled share links:

• Disabled share links continue to display and work (data can be submitted) in the current logged-in session.
• Disabled share links display a vague error message to the end user:

Error

    AssertionError: Insufficient permission to get the form
    AssertionError: Insufficient permission to get form source info

The critical bug is odd, and seems to relate to DB roles and public form share URLs. The public share URLs for some forms will return a 500 error: Here are the reproduction steps:

1. Run Mathesar 0.5.0 locally with docker run -it --name mathesar-050 -p 8000:8000 mathesar/mathesar:0.5.0
2. Create a new DB and install the bike shop schema.
3. Create a new form for the “customers” table.
4. Generate a public url for the form.
5. Open the public url in the same browser window as your logged-in user.
6. Observe that you can view and submit the form.
7. Open the public url in an incognito window, different browser, or after logging out of Mathesar.
8. Observe a 500 error on the public page.

Here are the first frames of relevant log output when attempting to view the public form page:

modernrpc.exceptions.RPCException: [-28009] DoesNotExist: Form matching query does not exist.
  at /code/mathesar/rpc/exceptions/handlers.py:25 in _raise_generic_error
  caused by /code/mathesar/rpc/forms.py:268 in get
           -> form_model = get_form(form_token, user)

modernrpc.exceptions.RPCException: [-31002] AssertionError: Insufficient permission to get the form
  at /code/mathesar/rpc/exceptions/handlers.py:25 in _raise_generic_error
  caused by /code/mathesar/utils/forms.py:145 in get_form
           -> assert has_permission_for_form(user, form_model), 'Insufficient permission to get the form'

modernrpc.exceptions.RPCException: [-31002] AssertionError: Insufficient permission to get form source info
  at /code/mathesar/rpc/exceptions/handlers.py:25 in _raise_generic_error
  caused by /code/mathesar/utils/forms.py:152 in get_form_source_info
           -> assert has_permission_for_form(user, form_model), 'Insufficient permission to get form source info'

We'll need to figure out what is causing this, particularly because it occurs on our default Docker setups--the quick command and the docker compose based installation. Curiously this is not an issue on the develop branch which is likely why it wasn't caught sooner.

@Anish Umale and @Pavish Kumar Ramani Gopal, can you please look into this critical issue as soon as possible? Concerning the non-critical ones as well, please let me know if you think they're higher priority than I've assessed. Otherwise I'll create new GitHub issues. 

Thanks,

Zack

--

Zack Krida

Product & Community Lead | Mathesar

za...@mathesar.org | Schedule a call

[Anish Umale]

• Disabled share links continue to display and work (data can be submitted) in the current logged-in session.
• Disabled share links display a vague error message to the end user:

Error

    AssertionError: Insufficient permission to get the form
    AssertionError: Insufficient permission to get form source info

From a purely backend perspective this is exactly what we expect functionality wise.

We do expect the logged in user to be able to submit the form iff they have a role_map on the relevant database even when the form isn't publicly shared.

The reason why the link keeps working is because the link is built on the token associated with the form and anybody with the knowledge of the form token can piece together a link `/share/forms/<form_token>/` and access the public share page even if the form isn't publicly shared.

RE: The critical bug, I did some digging and found that running the container with DEBUG=True makes the bug go away. This makes it kinda tough to debug as there are no error logs for the 500. I'm signing off for now, will continue investigating this tomorrow.

[Pavish Kumar Ramani Gopal]

The critical issue:

• I suspect that the cause is a missing trailing slash in the public share URL.
• The frontend does not append a trailing slash when copying the url.
• Eg., `http://localhost:8000/shares/forms/27325ed9-d5b8-4d4a-b645-d08ffe1c03f3`
• When opening this URL while the user is authenticated or when in debug mode, Django seems to redirect it correctly to the version with the trailing slash.
• `http://localhost:8000/shares/forms/27325ed9-d5b8-4d4a-b645-d08ffe1c03f3` -> `http://localhost:8000/shares/forms/27325ed9-d5b8-4d4a-b645-d08ffe1c03f3d/`
• (Notice the trailing slash)
• When requesting in an anonymous session, Django does not redirect it to the version with the trailing slash. This results in the 500 error.
• However, the error in your mail mentions that the form does not exist, which is odd. I haven't been able to reproduce that exact error. @Zack Krida Could you test this again and confirm that it works when the trailing slash is present?

> Disabled share links continue to display and work (data can be submitted) in the current logged-in session.

We removed the UI for authenticated internal forms, however, the backend still supports it. 

• We could make the `submit` action work only if the form is publicly shared.
• This would remove permission related issues (currently the form is tied to a role and any authenticated-user with access to the form's database would be able to fill in the form with that role).
• I think we should do this before the release.

• Retrieving the form on the backend is valid, as Anish mentioned above, since the user has access to the database. I could add a check on the frontend to render a 404 if the form is not publicly shared.
• I don't think this is a priority for the release, but a nice-to-have.

[Pavish Kumar Ramani Gopal]

Regarding the issues with trailing slashes in Django, I think we should investigate them and fix them for good (not for this release, but for the next one). I've been noticing them occasionally.

Django has an `APPEND_SLASH` setting which is True by default. However, it only works after all the URLs are checked and if the status is a 404. I suspect the issues we're seeing would be fixed if we ensure that the correct status code is returned.

1. Instead of returning a 500, we should be returning a 404 if there's a url mismatch.

2. We might be better off if we updated our URL patterns to accept urls with and without trailing slashes. 

[Pavish Kumar Ramani Gopal]

I've raised a PR Fix QA issues - 0.5.0 #4727 which should fix:

• The critical issue with public share URLs for some forms returning a 500 error, and
• The issue with disabled share links continuing to display and work (data can be submitted) in the current logged-in session.

[Zack Krida]

The fix works great. It's always the trailing slashes with Django, isn't it...

I'll create a ticket for addressing trailing slashes generally throughout Mathesar. 

Thanks again!

[Zack Krida]

The 0.5.0 Release Notes are ready for review. 

I'd still like to write a dedicated doc for Forms, which I'll do next in a separate branch for ease of review.

Best,

Zack

[Zack Krida]

Hi all,

Mathesar 0.5.0 is live and we've posted the announcement to Bluesky, Twitter, LinkedIn, Matrix, Reddit, and our newsletters. Thank you to everyone who contributed to the release!