Prioritizing projects

[Kriti Godey]

The Mathesar team met earlier today to discuss priorities and workflow now that we've launched. Meeting notes are here.

This thread is to discuss and decide on projects that we'll be working on.

Mathesar team: Please respond to this email with your thoughts on:

(1) what we should prioritize

(2) anything you're interested in owning or contributing to (specify which)

(3) any other project ideas that you think make sense to prioritize given our goals.

The list of projects is below.

- Funding - high priority
- Sponsorships
- Localization of UI - contractual obligation
- SaaS version of Mathesar
- Demo server improvements - low priority
- Removing SQLAlchemy
- Using Postgres users for users & permissions
- User growth & understanding user feedback
    - Decide on metrics for user growth
        - Should be driven by questions we have
    - We need to have a more specific project here (e.g. "grow Mathesar to X users")
- Installation improvements
    - We need to have a more specific project outcome here
- Community improvement projects
    - e.g. GSoC org admin
    - e.g. go through old issues and update them to be relevant from contributors
    - e.g. revamp community team with clear roles & responsiblities for each type of community member
    - e.g. document frontend & backend codebase structure
    - e.g. establish process for issue & PR triage
- Individual features on our roadmap

[Kriti Godey]

I'll start.

I'd like to own:

- Funding

- Release 0.1.1 (semi-project)

I need a frontend and a backend person to contribute to release 0.1.1.

I also own the GSoC org admin by default right now, but if someone wants to take that over, that would be helpful. I can explain what's needed if there are volunteers.

Pretty much everything else on the list is also up for grabs for other people to own. I do think we should stop work on the demo.

[Sean Colsen]

What we should prioritize

1. Funding
2. Removing SQLAlchemy

I’m interested in contributing to

• Funding

  • I’d like to be on-hand for helping with this project however I can.
  • I’d like to continue helping with “Sponsorships”, which I don’t think is large enough to warrant its own project. I see it as falling within “Funding”.

I’m interested in owning

• Chart & graph visualizations
• Visual representation of relationships between data
• Support for forms

(To be clear: I’m interested in owning any of these, not all of these.)

Other project ideas

I’d be interested in owning/contributing to any these as well.

• Front end store refactor

  We’ve discussed this a bit already. The work here would unblock/simplify some tickets such as:

  • Table inspector closes, then re-opens in the midst of performing some actions
  • Destroy TabularData in RecordSelector, TablePage, and TableWidget to prevent Memory leaks
  • Keep records in store while fetching
  • etc

• Usability improvements

  This would be a project where we’d lump a bunch of smaller tickets together from the Backlog, such as:

  • Allow content in selected cells to be copied and pasted elsewhere
  • Allow dragging to select cells from active cell
  • Cell selection is stuck in drag mode after closing the record selector
  • Allow users to add/remove filter conditions via the column header menu
  • Persist column widths via API
  • Make all sidebar panes resizable
  • Persist sidebar widths in localStorage

​

[Dominykas Mostauskis]

I see a few problems/improvements I'd like to see resolved on the backend. I ordered them into two lists. Two lists, because they can be worked on in parallel (though ideally we'd work on the service layer list first, imo). The order of the projects is such that later projects build upon those preceding it.

## Service layer list:

1. Reduce our Django code; use database identifiers on the frontend;

2. Use only Postgres permissions for permissions;

3. Replace SQLAlchemy with Psycopg3;

4. SaaS version.

Rationale being that before we start our work on the SaaS version, we want scalability. We need to replace SA to get scalability. And, we want to get rid of our custom permission code and unnecessary django state to make it easier to replace SA.

## Db layer list:

1. Organise our SQL code into Mathesar's Postgres extension;
2. Try moving logic from backend to Postgres extension.

Rationale being that managing our custom SQL code (like Postgres functions and types) ad-hoc is unsustainable (upgrades), we want the Postgres extension tooling for that. Once we have that, we want to try moving logic from our db namespace to the Postgres extension. Logic that would be simpler and/or more efficient on Postgres than in Python (type inference, column moving, table splitting, etc.).

## What I want to work on

I like all of these, but feel like I have more to offer in the service layer tasks. To narrow it down, I'd like to work on reducing Django code (project 1) and replacing SQLAlchemy (project 3). Experience with one would provide insights with the other. Project 2 (using Postgres permissions) would also provide insights for project 3, but I have less experience thinking about permissions.

I think that in the past in situations like these we took the opportunity to split similar work between multiple people, significantly degrading efficiency and outcome of both. I hope we're more careful with that in the future.

[Rajat Vijay]

## I am interested in contributing to the following things:

1. Frontend code documentation - already working on the first draft.

2. Localization of the UI.

3. SAAS version of matches

4. Cell Selection related issues that are already reported: https://github.com/centerofci/mathesar/issues?q=is%3Aissue+is%3Aopen+selection+label%3A%22restricted%3A+maintainers%22

5. Support for database view

## I think the order should be:

1, 2, 3, 4, 5 - but open to discussion on this.

[Brent Moran]

I'd like to own:

- Removing SQLAlchemy

I'd like to own or contribute:
- Using PostgreSQL Users for users and permissions
- GSoC

I'd like to contribute, but prefer not to own:
- Installation improvements (I think this should include Upgrades)
- Funding (my TZ might make this difficult; still I'm interested)
- User growth and understanding feedback

Prioritization:
- Funding (duh)
- I think removing SQLAlchemy should be soon. This should speed up
velocity, reduce bugs, and improve performance.
- Postgres users for users and permissions

[Pavish Kumar Ramani Gopal]

Prioritization:

I think we should prioritize funding and projects that block interested users from using Mathesar. Mainly these,

- Funding

- Installation improvements

- Users (Postgres/SSO) & Permissions improvements

Contribution:

I am fine with contributing to any of them. I don't have a strong preference for particular projects.  

Owning:

I am good with owning any of them, except the funding and Community improvement projects. I would like to contribute to both though.

[Mukesh Murali]

Prioritization:

- Funding
- Removing SQLAlchemy

- Installation improvements

I'd like to own or contribute:

- Installation improvements

- Using PostgreSQL Users for users and permissions

- Document frontend & backend codebase structure

- SSO (single sign-on) support

- Project templates

I'd like to contribute:
- Funding

- User growth and understanding feedback

[Anish Umale]

I'd like to contribute to:

- Removing SQLAlchemy

- Using PostgreSQL Users for users and permissions

- Documentation of backend code structure

- I'd like to help with the release if needed

I don't have any strong preferences on what should be prioritized. I think the priorities proposed by others make sense. 

[Ghislaine Guerin]

Priorities:
- Funding.
- Installation improvements.
- Using Postgres users for users and permissions.

In terms of projects that I am interested in owning or contributing to, I would like to focus on collaboration-related projects, specifically:
- Creating and sharing forms based on tables.
- Embedding tables and explorations in other websites.
- Sharing tables and explorations publicly.
- Real-time collaboration.

[Kriti Godey]

Thanks everyone.

I've created a Projects table on our staging server (not publicly accessible) with all this information: https://staging.mathesar.org/mathesar_tables/8/tables/9179/

### Notes:

- I added columns to track effort and impact. Some items are marked as impact "unknown" because we don't know what the use case is for those items. Any project involving those would first need to investigate that.

- I added notes to some items. e.g. I don't think we should do moving SQL code to a Postgres extension, because having a custom extension will mean Mathesar cannot be deployed on managed Postgres servers e.g. AWS RDS.

- Pavish, I didn't add you to any "interested" list because you said you were interested in everything.

- Removing SQLAlchemy and installation improvements are more like themes, not projects. The initial step here should be a short project that breaks down this work into smaller pieces.

Could everyone double check that the table makes sense and make updates as needed? Please feel free to add yourselves to the interested list too.

Once that's done, I'll send an email with next steps.

[Rajat Vijay]

I have reviewed this.

My major focus was on the Name, Impact, Effort, and Interested columns.

Removed me from the interested list of the "Usability Improvements" since there is already a FE team member there but can always come back if required, and added at a couple of other places.

[Sean Colsen]

Looks good to me

One thought:

• “Grow Mathesar to X users” does not seem like a project because: (a) it’s framed as a goal rather than an action, and (b) we still don’t know how to measure our user base. If we can re-frame this in terms of the work we’d do, then I think it could be a project.

​