What if Prometheus to Scape Anything from Anywhere with embedded Zero Trust?

[Rudford Hamon]

Hi everyone, 

My name is Rudford "R1". My son name is Rudford as well. We call him R2 like R2/D2. :) 

Although I am new to the Prometheus community, I am no stranger to getting involved, helping people, and pushing towards a better served mission for all e.g. Rotary International. 

What if Prometheus to Scape Anything from Anywhere with embedded Zero Trust? 

So, I am part an open source project called OpenZiti, where you can embed zero trust networking into anything (apps-to-apps, server-to-apps, server-to-server, etc) and be completely invisible while using basic community internet. VPNs, Bastions, or jump servers, including old school firewalls are NOT required. 

At OpenZiti, we use Prometheus and love the project just as much as everyone else. Since we love embedded zero trust security and scrapping data via Prometheus, we did a "zitification" test to see what the world would look like if Prometheus was able to do its magic with embedded zero trust and be completely invisible and scrape anything/anywhere without inherently risky vulnerabilities. 

Is this a topic that can be shared during the town hall meeting on May 30th? Would love learn, get feedback, and help contributing together towards making any and everything accessible and SAFE for ALL. 

I added it to the agenda, but as a newbie, and courtesy for the community, I thought I would ask and share it with all. 

Thanks everyone. 

R1

   

 

[Bjoern Rabenstein]

On 23.05.22 15:24, Rudford Hamon wrote:
>
> So, I am part an open source project called OpenZiti, where you can embed
> zero trust networking into anything (apps-to-apps, server-to-apps,
> server-to-server, etc) and be completely invisible while using basic
> community internet. VPNs, Bastions, or jump servers, including old school
> firewalls are NOT required.
>
> At OpenZiti, we use Prometheus and love the project just as much as
> everyone else. Since we love embedded zero trust security and scrapping
> data via Prometheus, we did a "zitification" test to see what the world
> would look like if Prometheus was able to do its magic with embedded zero
> trust and be completely invisible and scrape anything/anywhere without
> inherently risky vulnerabilities.

I'm not an expert in network security, so please pardon my possibly
imprecise use of jargon, but it sounds to me OpenZiti is a VPN where
you link the VPN parts directly into the software using the VPN.

The Prometheus project traditionally hasn't even tried to address
network security. We decided to delegate it to other components,
partially because Prometheus is already complex enough, partially
because we, as an OSS project, lack capacity and qualification to deal
with the network security aspects. We went as far as even refusing TLS
to be part of Prometheus components. Since TLS is so ubiquitious by
now and essentially seen as part of the network stack, we eventually
decided to support TLS directly rather than asking our users to set up
revers proxies, sidecars, etc. to add TLS support.

The latter gives you an idea what the threshold is where we would
consider linking network security related code directly into the
upstream projects.

Our users have very different approaches how to secure their networks
and how to organize metrics scraping, and I believe that will be the
case for the foreseeable future. (I should mention here that
cross-cluster scraping is considered a rare exception in the general
Prometheus deployment model.) Many might prefer a modular solution
that doesn't require changing all involved binaries with an SDK.

A "zitification" of the upstream Prometheus server (and presumably all
the other components of the Prometheus stack) seems to serve a fairly
niche une case at this moment. You are of course free to offer
"zitified" components, but as long as OpenZiti isn't even remotely as
ubiquitious as TLS, I cannot really imagine 1st class support in the
upstream Prometheus repositories.

That's just my initial thoughts based on a possibly incomplete
understanding of OpenZiti. Happy to hear the thoughts of other
Prometheus developers and of course more explanations from your side.

--
Björn Rabenstein
[PGP-ID] 0x851C3DA17D748D03
[email] bjo...@rabenste.in

[Rudford Hamon]

Re: [prometheus-developers] What if Prometheus to Scape Anything from Anywhere with embedded Zero Trust?

I appreciate your response and very thoughtful feedback. 

First, don't worry about the jargon. The principles and methodology of embedded zero trust security with OpenZiti is the same as protecting your most precious assets inside your house and/or apartment with software. Everything is inherently risky. 

To answer your questions: 

1. VPNs: VPNs are assigned to an IP address that can be scanned and exploited by bad actors; which is the #1 attack vector costing the technology industry $1 Trillion USD. It is equivalent to you waving a Flag outside your house/apartment and inviting a criminal to search if you have any openings (doors, windows, and garage, etc.) "open firewall ports" to steal your most precious assets. This is the same situation for Prometheus users. In order for them to scrape anything,  they "the house/apartment owner" have to open their living space, including basic security "firewall ports" in order to provide information for monitoring & observability. The issue is that firewalls and networks can't detect e.g. CVE's, zero day types of attacks, Log4J/Log4shell, etc. Solarwinds!

With OpenZiti, you don't have to expose your house/apartment address, open "firewalls ports", or use VPNs. How? OpenZiti enables you to connect any type of endpoint embedded with zero trust to live within a dark mesh fabric without any open firewall ports. Everything is invisible to the basic internet. All communications within the fabric are conducted with mutual TLS (mtls) and x509 certificates to authenticate, authorize, and connect. OpenZiti also has smart routing to control the internet weather (Figure 1, 2, 3). Cloud Native. Cloud Agnostic. Cloud zero Trust. 

2. Complexity: As you know, the more complex something is, the more insecure it becomes. Pain in the a**. The fundamental approach of OpenZiti is simplification for the most complex. Instead of trying to restructure, refactor, or etc. your existing infrastructure to embed zero trust security, you can start small like you would by inviting a guest to your house. Zero trust is a journey. For example, OpenZiti has a tunneler embedded with zero trust that can be used to connect (upstream or downstream) with a few lines of code. Super flexible and doesn't require any changes to your binaries. Also, as you mentioned, this will give the Prometheus family "end-users" an option to use whatever they feel is best for them. At least with the OpenZiti tunneler, the connection will be free with layer 7 security on the back-end. Once the Prometheus community becomes more familiar with embedded zero trust, the family may consider embedding zero trust at the application level so the project can scrape anything from anywhere without any exposers for both, Prometheus and end-users.

3.  Scrapping & distributed systems: The scrapping of cross-clusters was an example of granularity with OpenZiti. The methodology is applicable to any type of scrapping, including cross-cluster federated, which is pretty cool.  Also, when you look at Thanos, it's Prometheus on steroids that is cloud native and fully distributed, which makes it a perfect candidate to be future proofed with embedded zero trust. 

Most importantly, we love Prometheus and are glad that you and the family enabled us to join and to help make positive contributions for all. Open source love! 

Lastly, I'd like to get everyone's thoughts on the above, testing/trying it, and/or joint blogs or activities to talk about these types of critical security items to help the community with embedded zero trust security awareness. What do you think? 

Figure 1

Figure 2

Figure 3

Best, 

Rudford

[Bjoern Rabenstein]

On 01.06.22 11:28, Rudford Hamon wrote:
>
> For example, OpenZiti has a tunneler embedded with zero trust that
> can be used to connect (upstream or downstream) with a few lines of
> code. Super flexible and doesn't require any changes to your
> binaries. Also, as you mentioned, this will give the Prometheus
> family "end-users" an option to use whatever they feel is best for
> them. At least with the OpenZiti tunneler, the connection will be
> free with layer 7 security on the back-end. Once the Prometheus
> community becomes more familiar with embedded zero trust, the family
> may consider embedding zero trust at the application level so the
> project can scrape anything from anywhere without any exposers for
> both, Prometheus and end-users.

That sounds good. In that way, we can see what the adoption is without
requiring any changes on the Prometheus side.

[Rudford Hamon]

Yes :) What would be the best approach to see adoption and letting the community collectively know/try?  

Thanks,

Rudford

[Bjoern Rabenstein]

On 10.06.22 17:48, Rudford Hamon wrote:
> Yes :) What would be the best approach to see adoption and letting the
> community collectively know/try?

I guess you did the right thing already. A web search for "openziti
prometheus" gives tons of relevant results and discussions.

This list (prometheus-developers@) is aimed at the developers of
Prometheus (which seemed appropriate at first as the initial
discussion was around a zitified Prometheus binary). If you are more
interested in talking to _users_ of Prometheus (to help them with the
tunnel sidecar), the sister list promethe...@googlegroups.com
might be a better fit. And there are more community channels, see
https://prometheus.io/community/ .

Pitching a commercial product there is frowned upon, but as long as
you are sticking to an OSS project like OpenZiti, and your posts stay
relevant and to the point, I would assume it's OK to spread the news
via those channels.

[Stuart Clark]

On 14/06/2022 11:58, Bjoern Rabenstein wrote:
> On 10.06.22 17:48, Rudford Hamon wrote:
>> Yes :) What would be the best approach to see adoption and letting the
>> community collectively know/try?

> Pitching a commercial product there is frowned upon, but as long as
> you are sticking to an OSS project like OpenZiti, and your posts stay
> relevant and to the point, I would assume it's OK to spread the news
> via those channels.

The only things I'd say is to ensure you have the right expectations.
There might be some people on the list who are interested, but I'd
expect the vast majority probably don't have the time/interest/need for
such a solution. So you might have a few people asking for a bit more
information, but I wouldn't expect much to happen after your posting.

--
Stuart Clark

[Rudford Hamon]

Thank you for the feedback and help Bjoren @bjo...@rabenste.in. I only care about helping people and that's why I personally love OSS. 

Is there anything pending or plans that we can help contribute as developers to collectively align with the group? For OSS love, we just zitified Prometheus endpoints to help pull metrics there as well to keep bad actors from lurking to scan and exploit the community.

Here is the write up: https://openziti.github.io/ziti/metrics/prometheus.html