Adding Gitpod environments to prometheus/prometheus (as CI checks and for people to get started)

[Julius Volz]

Hi,

I got to know the people from Gitpod.io (https://www.gitpod.io/), and when I learned about their capability to not only use Gitpod as a regular online web-based dev environment, but to create ad-hoc environments based off PR branches as CI checks, I thought it would be a really neat thing to have that for Prometheus (it's free for OSS). It would also be generally great to give people a web-based container that has Prometheus pre-built and the dev environment ready and setup for working on the code. A proposed PR for the latter is here (AFAIK this doesn't do the CI integration yet, but that would be great too, then you can explore with the binary + code at a PR branch online, without checking it out locally): https://github.com/prometheus/prometheus/pull/7673

What do people think?

Cheers,

Julius

[Julius Volz]

+CC Sven from Gitpod

[Julien Pivotto]

Hello,

While getting new contributors is fundamental for any open source projects, I would be on the side to explore other possibilities than this one. Let me explain my concerns.

First of all, that is an extra tool. I am not a fan on adding extra tools on top of GitHub, like gerrithub or other review platforms. Especially tools that would not be used by us as a team.

When we redirect our users to a third party platform, we are 'endorsing' then in some way. I have not investigated the specifics of the proposed platforms, but we would have to review the tos of that platform and look if that is in line with our expectations.

We have got in the past first time contributors starting features, implementations, and then vanish before the code is in a mergeable state. We should find a compromise so that such a platform to embark more people does not lead us in that direction more often.

I also note that the scope is unclear. Is that for the react app? For TSDB? It would be strange to have a solution that would only work for one of them, e.g. react, because otherwise if someone comes with a solution for TSDB, we would end up with two solutions.

I do note that the open source version is limited to 50h/month; and that the application itself has < 1000 users according to GitHub.

--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CA%2BT6YoygV%3Dz3hDjb06DeK8E9ikT7RRe5tswnvvw_d%2Bv%2BOfnh2w%40mail.gmail.com.

[Augustin Husson]

Hello,

So far what I understood is that you have two things:

1. The possibility of coding online using a tablet

2. Having an online environment where your PR has been built. And like that you can play with the prometheus instance and test it smoothly.

( I know I reduced a lot the features provided )

I think the 2nd point is maybe the more interesting from a developer point of view. Since you will be able to see from the PR if the expectations brought are achieved. Of course it's easier to check it when it concerned the UI. But it could be used to check more tricky things such as the tsdb.

IMHO this is an old dream I never see working properly. Meaning having a dedicated environment created for each PR. And it seems they did it. It would be super cool to have it.

If I was able to have it the private company I would be so happy since it would make it super easier when reviewing the PR. ( Because you won't have to pull the PR + build it + launch it + test it. Which takes time, and if you have a huge amount of PR, then you just skip these parts usually, trusting that is working like it was previously designed).

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAFJ6V0rLMnJqju_7fOHPACJMGwdw8m0-ePdvLsDJUHFbXQNVAg%40mail.gmail.com.

[Julien Pivotto]

The rules, TSDB, scrapes, are not that easy to test in such an environment. They often require external exporters. Even locally the tests are the easiest way to do that. Service discovery requires service discovery setups.

The part where that is easy is the react UI which we already deploy on netlify.

[Augustin Husson]

On the tsdb part, yes I'm inline with you Julien in a sense where it's complicated to test it.

But I think here you could start with a light environment and then improve it later to check much more things.

Besides I would be a bit surprised than you cannot install some exporter and a Consul server for the discovery.

@Sven: I suppose that's possible doesn't it ?

Of course you won't cover every possibility every test for the tsdb and of course it won't cover the benchmark test, but that's a start. 

You are the expert here so you will tell me if I'm completely wrong. But I guess you have a sort of checklist when you are fast testing the tsdb from PR. And I suppose for most of these test it doesn't require a complicated dev environment.

And instead of pulling the PR, compiling the binary, potentially starting the different exporter, server needed for the discovery and so on, well you have it online. So potentially you will gain time.

Le lun. 27 juil. 2020 à 09:07, Sven Efftinge <sv...@gitpod.io> a écrit :

Hey all,

Just chiming in to let you know that I'm happy to answer any questions you have. 

FWIW Gitpod requires just the config file and the app (for CI), so it doesn't have a lot of footprint for those who absolutely don't want to use it.

Sven

[Julius Volz]

I think having Gitpod as a CI check wouldn't just be useful for manual testing, but it's nice for a reviewer to be able to jump into a ready-to-go checked-out-and-built dev environment based off a PR branch, with editor and code intelligence that allows you to properly jump around and understand the PR code, etc., without having to check out the PR branch locally. For people who want it, they can even comment on changes directly from Gitpod (https://www.gitpod.io/docs/code-reviews/).

If at any point Gitpod stops working / we don't like it anymore, we can just remove it. I don't think we would ever hit the 50h limit, and even then that's probably negotiable.

[Sven Efftinge]

If at any point Gitpod stops working / we don't like it anymore, we can just remove it.

Absolutely! Also please ping me or someone from Gitpod if you have issues or any kind of useful feedback.

 

I don't think we would ever hit the 50h limit, and even then that's probably negotiable.

We offer a free unlimited hours plan for professional open source developers, which you can request once you hit the 50h/month limit.

[Sven Efftinge]

Besides I would be a bit surprised than you cannot install some exporter and a Consul server for the discovery.

@Sven: I suppose that's possible doesn't it ?

You can install almost anything that runs on Linux. 

If you provide me with a link to some more information of what is needed I can take a look.

[Julius Volz]

I think it could be a really useful addition, both for new contributors, as well as for reviewers / maintainers.

Julien, are you still opposed to this in principle? Any opinions by other team members?

[Julien Pivotto]

On 04 Aug 20:47, Julius Volz wrote:
> I think it could be a really useful addition, both for new contributors, as
> well as for reviewers / maintainers.
>
> Julien, are you still opposed to this in principle? Any opinions by other
> team members?

I have read the TOS and nothing frightened me. I am not blocking this
initiative, you can go ahead.

>
> On Tue, Jul 28, 2020 at 11:30 AM Sven Efftinge <sv...@gitpod.io> wrote:
>
> >
> >> Besides I would be a bit surprised than you cannot install some exporter
> >> and a Consul server for the discovery.
> >> @Sven: I suppose that's possible doesn't it ?
> >>
> >
> > You can install almost anything that runs on Linux.
> > If you provide me with a link to some more information of what is needed I
> > can take a look.
> >
> >

--
Julien Pivotto
@roidelapluie

[Julius Volz]

Great, ok :)

I'll go ahead and work with Sven to get some Gitpod integration going then for prometheus/prometheus as a start, unless someone else objects.

[Julien Pivotto]

I have realized that this kind of feature will also come from github directly:

https://github.com/features/codespaces/

> --
> You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CA%2BT6Yoz9NrnnEqQEajPzYQ6KyRsK1tKCZ6nBH3LhpVUX2%2BJofA%40mail.gmail.com.

--
Julien Pivotto
@roidelapluie

[Sven Efftinge]

Yes, partly. E.g. codespaces are not pre-building your branches [1], so you are going to wait every time you open a fresh environment.

As a result people will probably not open fresh ones, but go back to old environments and maintain/sync them manually, therefore, ending up in the same yak shaving and configuration drift experience we know from local dev environments. But it's a great start in the right direction for sure as at least the initial onboarding is automated as well.

[1] - https://www.gitpod.io/blog/prebuilds/

[Julien Pivotto]

It also seems that codespace is tight to the user, not to the repo. e.g.
the user will pay and will be able to use that in all their
repositories.

So the repo who will implement the helpers for codespace will actually
sell a paying feature of github...

> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAEzbUNXR438qmmiaMM_xZ6BOR7ZFwq7R3AYQBOQcifL0d8LGDw%40mail.gmail.com.

--
Julien Pivotto
@roidelapluie

[Julien Pivotto]

I did try gitpod on this PR (apparently the app is installed now?):

https://github.com/prometheus/prometheus/pull/7820

And it throws and error in the react UI when I launch a Prometheus
query.
It works when I open the react UI in a new tab.

I am quite wondering how new contributors would deal with that kind of
issues. I don't know if the bug is in our code, or in gitpod.

On 14 Aug 08:17, Sven Efftinge wrote:

[Sven Efftinge]

I tried to reproduce it, but it never happened for some reason. 

Gitpod doesn't do anything special, so this seems to be a timing issue with the webpack setup.

Did you try to reproduce that a second time?

[Julius Volz]

Trying that out, I also get an error, but only *sometimes*. The majority of queries run ok, and then sometimes I run one (not even the first one after page load) that ends in an error like this (based on https://gitpod.io#github.com/prometheus/prometheus/pull/7820):

I haven't managed to create the same error in the Netlify deploy preview for the same PR yet.

[Julien Pivotto]

On 18 Aug 11:53, Julius Volz wrote:
> Trying that out, I also get an error, but only *sometimes*. The majority of
> queries run ok, and then sometimes I run one (not even the first one after
> page load) that ends in an error like this (based on
> https://gitpod.io#github.com/prometheus/prometheus/pull/7820):
>
> [image: error.png]
>
> I haven't managed to create the same error in the Netlify deploy preview
> for the same PR yet.

This seems a different error, the error I have is from inside the
editor, in the tab.

> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CA%2BT6Yoyqtc_PY2U0ZN2jDrFtPCB0j_p_Haq07sspCPOKBtjUhw%40mail.gmail.com.



--
Julien Pivotto
@roidelapluie

[Sven Efftinge]

On Tue, 18 Aug 2020 at 11:55, Julien Pivotto <roidel...@prometheus.io> wrote:

On 18 Aug 11:53, Julius Volz wrote:
> Trying that out, I also get an error, but only *sometimes*. The majority of
> queries run ok, and then sometimes I run one (not even the first one after
> page load) that ends in an error like this (based on
> https://gitpod.io#github.com/prometheus/prometheus/pull/7820):
>
> [image: error.png]
>
> I haven't managed to create the same error in the Netlify deploy preview
> for the same PR yet.

This seems a different error, the error I have is from inside the
editor, in the tab.

That preview really is the website as is rendered in an iframe. So it renders the same no matter you open a new tab or use the preview tab on the right side.