UI dependencies and stop the codemirror-promql synchronisation

[Augustin Husson]

Dear Prometheus developers,

As you may know, since a couple months ago, we changed the way the UI is organized to be able to include the project codemirror-promql directly in Prometheus. The idea was when changing the PromQL syntax, we also modify codemirror-promql so we don't have any desync between the UI and the backend. 

We are using the npm workspace feature which is the way with npm to manage a mono-repo, and since this moment we have some troubles to upgrade the UI dependencies like:

• Not every files are correctly updated by dependabot
• There are some conflicts between the dependencies used in codemirror-promql and in the react-app. Basically a dependency A in codemirror-promql could be upgraded, but cannot in the react-app because the react-app has some constraints that prevent this upgrade. Which leads to not necessarily knowing which version of the dependency A is used.

With just these two points above, updating the UI dependencies require too much knowledge about how the npm workspaces are working and it is hard to automatize.

Since the last version where @Bjoern Rabenstein faced some difficulties to upgrade the UI dependencies, I have been thinking about how to improve the situation. I have compared the Prometheus repo with what we are doing in Perses where it is working without any issues (aka using a npm monorepo).

And my conclusion is actually: we didn't embrace the way the monorepo is supposed to work in the UI way.

What we did is to put a side project (codemirror-promql) alongside of the react-app without changing a thing. But instead we should actually align and put in common a certain amount of dependencies used across the different UI modules.

It wasn't possible to align the dependencies because we (actually I) said, it will be cool to synchronize codemirror-promql to the repo prometheus/codemirror-promql where we will make the release. 

And for that, we need to build the repo which requires some dev-dependencies ...etc so that cannot be put in common with the react-app. 

And like that we still have a tag associated with the release of codemirror-promql.

So after this tentative to explain the situation, here what I would like to do, and where I would like your opinion on:

I would like to stop the synchronization of the codemirror-promql. So we need to find a new way to release codemirror-promql directly in the Prometheus repository.

So we could say, it is released at the same time as Prometheus itself. And then:

• With the same version. So we will bump the version of codemirror from 0.19.0 to 2.35.
• Its own version (with a sort of script to know if codemirror-promql has to be released since the Prometheus release or not).
  

In both cases, the release of codemirror-promql will be behind the unique tag of Prometheus. (i.e. v2.X.Y)

Once we find the way, I will make the changes and after a couple of PRs we should be back to a situation where with a quick command we will be able to update or upgrade the UI dependencies. (The command is already known and I experimented in Perses).

Sorry for the long email, I tried to explain why it's a bit hard to upgrade the dependencies and it's actually quite hard to explain without entering in too much detail.

All the best,

Augustin.

[Julien Pivotto]

On 17 Mar 14:05, Augustin Husson wrote:
> Dear Prometheus developers,
>
> As you may know, since a couple months ago, we changed the way the UI is
> organized to be able to include the project codemirror-promql directly in
> Prometheus. The idea was when changing the PromQL syntax, we also modify
> codemirror-promql so we don't have any desync between the UI and the
> backend.
>
> We are using the npm workspace feature which is the way with npm to manage
> a mono-repo, and since this moment we have some troubles to upgrade the UI
> dependencies like:
>

> - Not every files are correctly updated by dependabot
> - There are some conflicts between the dependencies used in

> codemirror-promql and in the react-app. Basically a dependency A in
> codemirror-promql could be upgraded, but cannot in the react-app because
> the react-app has some constraints that prevent this upgrade. Which leads
> to not necessarily knowing which version of the dependency A is used.
>
> With just these two points above, updating the UI dependencies require too
> much knowledge about how the npm workspaces are working and it is hard to
> automatize.
>

> Since the last version where @Bjoern Rabenstein <bjo...@rabenste.in> faced

> some difficulties to upgrade the UI dependencies, I have been thinking
> about how to improve the situation. I have compared the Prometheus repo
> with what we are doing in Perses where it is working without any issues
> (aka using a npm monorepo).
> And my conclusion is actually: we didn't embrace the way the monorepo is
> supposed to work in the UI way.
>
> What we did is to put a side project (codemirror-promql) alongside of the
> react-app without changing a thing. But instead we should actually align
> and put in common a certain amount of dependencies used across the
> different UI modules.
> It wasn't possible to align the dependencies because we (actually I) said,
> it will be cool to synchronize codemirror-promql to the repo
> prometheus/codemirror-promql where we will make the release.
> And for that, we need to build the repo which requires some
> dev-dependencies ...etc so that cannot be put in common with the react-app.
>
> And like that we still have a tag associated with the release of
> codemirror-promql.
>
> So after this tentative to explain the situation, here what I would like to
> do, and where I would like your opinion on:
>
> I would like to stop the synchronization of the codemirror-promql. So we
> need to find a new way to release codemirror-promql directly in the
> Prometheus repository.
>
> So we could say, it is released at the same time as Prometheus itself. And
> then:
>

> - With the same version. So we will bump the version of codemirror from
> 0.19.0 to 2.35.
> - Its own version (with a sort of script to know if codemirror-promql

> has to be released since the Prometheus release or not).
>
> In both cases, the release of codemirror-promql will be behind the unique
> tag of Prometheus. (i.e. v2.X.Y)
>
> Once we find the way, I will make the changes and after a couple of PRs we
> should be back to a situation where with a quick command we will be able to
> update or upgrade the UI dependencies. (The command is already known and I
> experimented in Perses).
>
> Sorry for the long email, I tried to explain why it's a bit hard to upgrade
> the dependencies and it's actually quite hard to explain without entering
> in too much detail.
>
> All the best,
> Augustin.

Hello Augustin,

I agree that the current way of working is painful. I'd be in favour of
having codemirror living its own lifecycle outside of the main
repository.

It will also be easier for downstream builders. NPM workspaces are not
supported everywhere yet (see
https://github.com/NixOS/nixpkgs/pull/156078).

As far as I am concerned, we could implement the change ASAP.

Regards,

--
Julien Pivotto
@roidelapluie

[Augustin Husson]

Hello Julien,

I'm not sure I'm following what you would like.

Do you mean you would like to fallback to the situation before we had codemirror into the upstream repo ? And so having only the react-app in Prometheus ? (which removes the requirement to use the nmp workspaces feature).

That's another way yeah, but that would be a regression from my point of view since you will have to ensure again that codemirror-promql is correctly synchronized with the changes of PromQL.

Probably releasing codemirror-promql with the same tag and same version of Prometheus is the simplest way to release it and will allow us to put in common the different deps used between codemirror-promql and the react-app.

It will create many releases with no changes for codemirror-promql, but that would be the garantie that the version v2.35.0 of codemirror-promql is compatible with the same version of Prometheus.

Regards,

Augustin.

[Julien Pivotto]

On 21 Mar 10:57, Augustin Husson wrote:
> Hello Julien,
>
> I'm not sure I'm following what you would like.
> Do you mean you would like to fallback to the situation before we had
> codemirror into the upstream repo ? And so having only the react-app in
> Prometheus ? (which removes the requirement to use the nmp workspaces
> feature).
>
> That's another way yeah, but that would be a regression from my point of
> view since you will have to ensure again that codemirror-promql is
> correctly synchronized with the changes of PromQL.
>
> Probably releasing codemirror-promql with the same tag and same version of
> Prometheus is the simplest way to release it and will allow us to put in
> common the different deps used between codemirror-promql and the react-app.
> It will create many releases with no changes for codemirror-promql, but
> that would be the garantie that the version v2.35.0 of codemirror-promql is
> compatible with the same version of Prometheus.
>
> Regards,
> Augustin.

It means we will have to make the release CI release codemirror first,
then Prometheus. How do we commit the codemirror release hash in the
package.lock, in a single commit/tag?

Regards,

> --
> You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAOJizGeX7aJ6k1-acEd264FT7m6txgU-N6_7gxGryOVoxqE2cQ%40mail.gmail.com.

--
Julien Pivotto
@roidelapluie

[Augustin Husson]

Bumping the codemirror version / release can be simply done in the PR to the release branches with the command `npm version "v2.35" --workspaces && npm install`.

And then during the release CI, we just have to add somewhere `cd web/ui/module/codemirror-promql && npm publish`

Does it answer your questions ?  

[Julien Pivotto]

On 21 Mar 11:15, Augustin Husson wrote:
> Bumping the codemirror version / release can be simply done in the PR to
> the release branches with the command `npm version "v2.35" --workspaces &&
> npm install`.
> And then during the release CI, we just have to add somewhere `cd
> web/ui/module/codemirror-promql && npm publish`
>
> Does it answer your questions ?

Okay, from your first email, my understanding was that codemirror was
going to be a versioned dependency of the UI.

I am fine with releasing both at the same time from the Prometheus
repository. I hope it will solve the package.json issues.

> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAOJizGcxb_Hy0_45H4Pv2CLnEASkGxNU_j77p0yF0nHmq-whyQ%40mail.gmail.com.

--
Julien Pivotto
@roidelapluie

[Augustin Husson]

I finally finished this work. It's available here: https://github.com/prometheus/prometheus/pull/10517.

With that we are able to update the dependencies with a script

[husson....@gmail.com]

Hello, 

I would like to give some news around this subject.

I have created a npm organisation called prometheus-io. It will be used to publish the npm packages provided by Prometheus such as codemirror-promql.

It means for the next version of codemirror-promql, you will need to install the package "@prometheus-io/codemirror-promql" instead of codemirror-promql. Same rule applied for lezer-promql, it will be available under the npm package "@prometheus-io/lezer-promql"

I have opened a PR that defines how these packages are going to be published in Prometheus. Basically every time Prometheus is released, you will have a release for these packages regardless there is a change in them.

Note that these new packages are not yet available. They will be published with the next Prometheus release (end of June).

Cheers,

Augustin.