Frequent captcha, cloudfare blocks and temporary cert problems

[Matthew Lynch]

I am running e2guardian 5.5.7r with MITM with self-signed cert.  It had been running well (perhaps an earlier version though) but then a while back I began to experience unusual behavior intermittently with some websites.  There are 3 separate problems

1. There will be an SSL cert error ERR_CERT_COMMON_NAME_INVALID

The "common name" of the cert is an ip address. It is almost as if the DNS resolves the ip and then replaces the web domain with the IP in the cert.  However, REFRESHING the browser will often fix the problem and load properly.  When I reduced the cpu load of the e2guardian computer, this SEEMED to improve, but I'm not sure because it still occurs frequently.

2. I am now frequently challenged by captcha to prove I am human, like quora, apkpure, ebay and many other popular websites.

3. I often received a "blocked" page from cloudfare.  Such as this one:

https://www.justanswer.com/car/bw062-looking-2017-honda-accord-sport-brake-specs.html

These problems are eliminated if I bypass the e2guardian via router.

Is anyone else experiencing these things?  Are they separate or related problems?

[Philip Pearce]

HI,

 

Are you using explicit or transparent proxy?   How e2guardian generates the COMMON_NAME in the certificate is different in each case, this information would be helpful to resolve this.

 

Have you turned off ECH and DoH support in your browser(s)?

 

Regards

Philip

 

--
E2guardian:
https://groups.google.com/d/forum/e2guardian
Github:
https://github.com/e2guardian/e2guardian
Follow us on twitter:
https://twitter.com/e2guardian
---
You received this message because you are subscribed to the Google Groups "e2guardian" group.
To unsubscribe from this group and stop receiving emails from it, send an email to e2guardian+...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/e2guardian/9abb1d3d-bb62-4b0f-a54a-38b8b721f5b3n%40googlegroups.com.

[Matthew Lynch]

I am using a transparent proxy.  I was trying to create a setup that avoided the need for specific browser or computer configuration, but I guess ECH/DoH is going to hamper that.   I also had a pi-hole for DNS but it no longer receives any queries.  I will try to experiment with ECH and DoH. 

[Philip]

Setting the addECHtoFlags option (v5.5.6 onwards) may help you to diagnose this.  It will add an E to the log flags when ECH is detected in the ClientHello (log types 7, 8 only).   So you could see if your issues are associated with ECH.

Regards

Philip

[Matthew Lynch]

I turned addECHtoFlags to on and log mode to 7 and it gives "PM" for sites that offer challenge and "PS" for sites that are trusted".  It does not appear ECH is detected. 

Now the government website pubmed.org is also now returning "403 forbidden" when I access through MITM proxy (with ext flags PM).  But if I add to exception list it works.

Is the server website detecting my MITM and flagging it as suspicious?

[Matthew Lynch]

I turned off ECH (I think) in the browsers but I still get the frequent captchas, but only if traffic is decyrpted by e2guardian.  E2guardian started causing PubMed, a government website (https://pubmed.ncbi.nlm.nih.gov/), to give "403 forbidden".  If I bypass e2guardian with router traffic rules or by adding the site to the exception list, I will avoid the captchas and PubMed works.    It seems like this issue is different from the ERR_CERT_COMMON_NAME_INVALID cert error.

This leads me to think that these websites are detecting the MITM and thinking it is suspicious.

I turned addECHtoFlags on (still using 5.5.7r) and I get log flags "8443:TM" for filtered sites and "8443:TH" for exception list sites.  There is no E flag.  

I don't know if it's my network setup not masquerading something it should be or something like that.  I have an OpenWrt router marking outgoing packets on 80 and 443 and re-routing them to the e2g host on 80 and 443, and then an iptable on the e2g_host routing to 8080 or 8443 respectively.

On Monday, April 7, 2025 at 5:14:44 AM UTC-4 Philip wrote:

[Matthew Lynch]

Another example: https://www.thelawnforum.com/ -> returns error 406 with e2guardian active:

1. Disable blocking but allow logging by setting reportinglevel = -1 -> still 406

2. Add to exception list -> works

3. Bypass e2guardian by router -> website works

4. Set sslmitm = off -> website works

This seems like a problem with the remote server seeing the traffic is MITM and rejecting it.

Setup is transparent proxy with MITM with e2guardian 5.5.7r.

Some websites (esp Cloudflare hosted) send captchas instead of sending an error.   Some of these like Quora have widely variable content that would need to be filtered so cannot be added as an exception.  It is also occurring fairly frequently so adding exceptions in a reactive way is not as practical.

Is there a way to make e2guardian server act like a web browser or in some other way seem less threatening to websites?

[Matthew Lynch]

Continuing testing of thelawnforum.com problem above:

5. Set browser to explicit proxy rather than transparent -> 406 error