Improve backend logging to improve bug reporting

[Dominykas Mostauskis]

We're experiencing some tough, hard-to-reproduce backend bugs. We discussed this with Brent and didn't come up with a better short term solution than to improve reporting by improving logging.

I think this bug report is pretty representative of what bug reports currently look like: https://github.com/centerofci/mathesar/issues/1791. We'd like a lot more context.

We're looking to setup logging to contain at least:

- HTTP requests and responses;

- HTTP request and response bodies (at a lower logging level);

- Information about critical code being visited (e.g. `reset_reflection`);

- and, whatever else you recommend.

We're talking about a quick and easy logging improvement to get better bug reports as soon as possible. We also discussed the usefulness of a generally good logging framework that we could rely on in the long term, but this discussion is not about that.

Help and input wanted:

- insights into how to best setup logging, especially in the context of Django?

- what should we log from the get-go?

Thanks

[Sean Colsen]

Being a front end dev, I can only see this problem from afar, but I have a gut feeling that our time will be better spent on the anti-state refactor than on continuing to chase down all these bugs. I think we need to get to the root cause. The recent refactoring was only supposed to be a band-aid solution, and I think it’s serving its purpose well enough for now. We should be careful not to spend too much time polishing that band-aid, because I think we’ll be better off ripping it off sooner than later. It’s not like adding logging can necessarily hurt. We’ll likely want some form of logging eventually. But I wouldn’t be surprised if logging ends up costing us some time (both to set up, and also to analyze) without producing significant improvements towards stability.

​

[Dominykas Mostauskis]

Is there agreement that we can live with current state-related bugs?

I agree that the recent state and performance refactors are band aids, and that future refactors (post release) are the real solution. Can we live with these bugs until then?

If not, we need better bug reports.

[Sean Colsen]

I can live with the current bugs, yes.

Qualitatively, it feels like there's been a uptick in back end bugs that are tricky to reproduce. But also it's not like we went from being super stable to being unstable. I've been encountering slippery back end bugs for the last year. Many times I don't report them because it's hard to provide reproduction steps. I've been pushing myself to report more bugs lately even when I don't have reproduction steps, just because I'm hoping that the information will at least raise awareness within the back end team that issues exist. I wouldn't say the product is significantly less stable now than it was before the recent refactoring. Rather, I'd say that it's always felt very unstable and recently there's been a marginal increase in that instability. That's why I would lean towards addressing the root causes.

[Dominykas Mostauskis]

> Is there agreement that we can live with current state-related bugs?

I'd like to hear whether others agree or disagree. I was under the (possibly false) impression that this is urgent, because the decrease in stability has been beyond marginal.

> Qualitatively, it feels like there's been an uptick in back end bugs that are tricky to reproduce.

The sad side effect of having a longer-lived state.

[Kriti Godey]

This is not urgent. I think it's worth spending time to fix individual bugs when they are reproducible using common operations in the frontend. If we can't reproduce a bug consistently, or if it involves a more esoteric set of actions, then it's not a priority.