API Requests being Blocked by Cloudflare Protections?

[Brian Pepple]

Hi,

Notice the API is throwing HTTP 403 error codes when attempting to make
requests with python, but is still accessible through a web browser. I'm
guessing it's due to your Cloudflare site protections, and if that is
the cause, are you able to whitelist the API endpoints?

Also, have you given any thought on using Basic Auth, in addition to the
Session-based Auth, since it's just a very minor change to the config to
enable (just need to add
"rest_framework.authentication.BasicAuthentication" to your
"DEFAULT_AUTHENTICATION_CLASSES" in the settings). That would make it
*much* nicer to use in a script.

Thanks,

/B

--
Brian Pepple <bpe...@metron.cloud>

https://about.me/brian.pepple

[Jochen G.]

Hi,

we had to turn on the under attack mode, which gives an interstitial on
every page. For web browsers, you do this once and are fine for a day,
which goes for the API as well.

While I see a way to turn on attack mode for specific pages, I don't see
to turn it off for specific pages.

Haven't looked at other auth so far.

Jochen

Am 23.06.25 um 14:48 schrieb Brian Pepple:

[Brian Pepple]

On 6/24/25 12:03, 'Jochen G.' via gcd-tech wrote:
> we had to turn on the under attack mode, which gives an interstitial
> on every page. For web browsers, you do this once and are fine for a
> day, which goes for the API as well.

I figured that was the case, but doesn't that prevent the any API
requests the are not signed in with a Session cookie, i.e. the
AnonReadOnly permission done thru a script?

>
> While I see a way to turn on attack mode for specific pages, I don't
> see to turn it off for specific pages.
>
> Haven't looked at other auth so far.
>

Your current permission implementation requires a session cookie to be
saved, which is fine if using a web browser, but if done by script it's
much more involved (since you need to access that Session cookie or make
a request for one, which I'm not sure is documented what the login URL
to do so would be).


Using Basic Authentication, which handled out the box with pretty much
any HTTP library (since it just a base64 encoded string of the user name
& password), means much less code needs to be written.


BTW, do you know how many folks are using the API thru something other
than a web browser?


Take care,

[Matthew Cox]

I can confirm- the API breaks when using a script, rather than a browser.

--
You received this message because you are subscribed to the Google Groups "gcd-tech" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gcd-tech+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/gcd-tech/c584823b-d9f8-457a-845a-db07a445d5c2%40metron.cloud.

[Jochen G.]

Am 24.06.25 um 20:06 schrieb Brian Pepple:

> On 6/24/25 12:03, 'Jochen G.' via gcd-tech wrote:
>> we had to turn on the under attack mode, which gives an interstitial
>> on every page. For web browsers, you do this once and are fine for a
>> day, which goes for the API as well.
> I figured that was the case, but doesn't that prevent the any API
> requests the are not signed in with a Session cookie, i.e. the
> AnonReadOnly permission done thru a script?

under attack mode, anon likely doesn't work

>> While I see a way to turn on attack mode for specific pages, I don't
>> see to turn it off for specific pages.
>>
>> Haven't looked at other auth so far.
>>
> Your current permission implementation requires a session cookie to be
> saved, which is fine if using a web browser, but if done by script it's
> much more involved (since you need to access that Session cookie or make
> a request for one, which I'm not sure is documented what the login URL
> to do so would be).

> Using Basic Authentication, which handled out the box with pretty much
> any HTTP library (since it just a base64 encoded string of the user name
> & password), means much less code needs to be written.

drf docs say to only use basic auth for testing, so I am not sure if we
go this path.

The login-url can now be accessed by a script, added a rule for that.

> BTW, do you know how many folks are using the API thru something other
> than a web browser?

No

Jochen

[Brian Pepple]

On 6/24/25 18:01, 'Jochen G.' via gcd-tech wrote:

drf docs say to only use basic auth for testing, so I am not sure if we go this path.

Yeah... you need to make sure to use some basic security measures (https-only, correct user/model permissions, sensible request methods offered, etc), and be aware of any potential attack vectors. Depending on your security needs, token access might be a better option, tho it also has some potential security deficiencies.

Most of the other comic sites use one of the following:

• Basic Authentication
• Metron
• Token-Based
• MangaDex
• MangaUpdates
• Comic Vine (which calls it an ApiKey, which it technically isn't since it's assigned to a user)
• ApiKey
• League of Comic Geeks
• OAuth
• AniList
• MyAnimeList
• JSON Web Token Authentication
• Kitsu
  

Each has it's pro and cons, but in general, the more secure the method the more work will be required to implement it.

The big question is how the consumers of your API plan to use it. If they plan to use it for an AJAX/JS/Browser project, it makes sense to only provide Session-based authentication, but if they plan to use it with a program/script you would want to use one of the other authentication methods.

[Jochen G.]

I turned on BasicAuthentication. Since we only do read-only API's, the
security aspects are not really an issue for us, but the user needs to
be careful.

Am 25.06.25 um 19:26 schrieb Brian Pepple:

> On 6/24/25 18:01, 'Jochen G.' via gcd-tech wrote:
>> drf docs say to only use basic auth for testing, so I am not sure if
>> we go this path.
>
> Yeah... you need to make sure to use some basic security measures
> (https-only, correct user/model permissions, sensible request methods
> offered, etc), and be aware of any potential attack vectors. Depending
> on your security needs, token access might be a better option, tho it
> also has some potential security deficiencies.
>
> Most of the other comic sites use one of the following:
>

> * Basic Authentication
> o Metron
> * Token-Based
> o MangaDex
> o MangaUpdates
> o Comic Vine (which calls it an ApiKey, which it technically isn't

> since it's assigned to a user)

> * ApiKey
> o League of Comic Geeks
> * OAuth
> o AniList
> o MyAnimeList
> * JSON Web Token Authentication
> o Kitsu

>
> Each has it's pro and cons, but in general, the more secure the method
> the more work will be required to implement it.
>
> The big question is how the consumers of your API plan to use it. If
> they plan to use it for an AJAX/JS/Browser project, it makes sense to
> only provide Session-based authentication, but if they plan to use it
> with a program/script you would want to use one of the other
> authentication methods.
>
> Take care,
>
> /B
>
> --
> Brian Pepple<bpe...@metron.cloud>
>
> https://about.me/brian.pepple
>

> --
> You received this message because you are subscribed to the Google
> Groups "gcd-tech" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to gcd-tech+u...@googlegroups.com <mailto:gcd-
> tech+uns...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/gcd-

> tech/e196e44f-4bf4-41c2-9286-8439aeb1ae92%40metron.cloud <https://
> groups.google.com/d/msgid/gcd-tech/
> e196e44f-4bf4-41c2-9286-8439aeb1ae92%40metron.cloud?
> utm_medium=email&utm_source=footer>.

[Brian Pepple]

On 7/4/25 17:32, 'Jochen G.' via gcd-tech wrote:
> I turned on BasicAuthentication. Since we only do read-only API's, the
> security aspects are not really an issue for us, but the user needs to
> be careful.

Thanks! Has this been pushed to production?


BTW, If you're worried about user security it might be worthwhile to use
a Token Auth, which is basically the same as Basic Auth, but does allow
the admin to quickly terminate access without affecting other sessions
(namely the website access).

[Jochen G.]

this is so far only on production awaiting confirmation that it works

Am 05.07.25 um 17:03 schrieb Brian Pepple:

[Brian Pepple]

On 7/8/25 05:20, 'Jochen G.' via gcd-tech wrote:
> this is so far only on production awaiting confirmation that it works

So it should be available for users, right now? Or am I misunderstanding
you (which is entirely possible).

[Jochen G.]

Did anyone try ?

I am also not sure if cloudflare blocks the API, but I changed something
there as well.

Am 08.07.25 um 20:49 schrieb Brian Pepple:

[Jochen G.]

For now disabled the enhanced cloudflare-protection, but likely will
turn that back on in a week while I am on vacation.

But, I did check the changes I recently did before turning it off, so
API-calls do pass the enhanced protection.

Jochen

Am 20.07.25 um 14:42 schrieb 'Jochen G.' via gcd-tech:

[Brian Pepple]

On 7/20/25 08:42, 'Jochen G.' via gcd-tech wrote:
> Did anyone try ?
>
> I am also not sure if cloudflare blocks the API, but I changed
> something there as well.
>

I checked the API docs and the Django settings (master branch)(1) and
didn't see that the authentication class is being defined, which in that
case, defaults to SessionAuthentication only.

If also think cloudflare would block the API without the endpoints
(/api) being whitelisted.


1)
https://github.com/GrandComicsDatabase/gcd-django/blob/dc6d0f5404da8b0a0064bfb29eb4106cc27ce963/settings.py#L341


Take care,

[Jochen G.]

Our public django settings don't necessarily reflect the production site.

As said, I changed the cloudflare settings.

Am 20.07.25 um 18:17 schrieb Brian Pepple:

[Brian Pepple]

On 7/20/25 12:20, 'Jochen G.' via gcd-tech wrote:
> Our public django settings don't necessarily reflect the production site.
>

Hmm... did a quick test of the Basic Auth of the API using posting (1),
and seem to have hit the API limit of 30/hr for unauthorized. Looking at
the header from the response it's still using Session-based
authentication. 🤷‍♂️

Does the server logs show that anyone been to access it with Basic
Authentication?

BTW, even though I've a bunch of questions regarding the API, I *do*
appreciate the work/time you've spent on it.


1) https://github.com/darrenburns/posting

[Jochen G.]

I have no idea in which logs this would be put from the django-side

Am 20.07.25 um 19:12 schrieb Brian Pepple:

[Brian Pepple]

On 7/20/25 15:48, 'Jochen G.' via gcd-tech wrote:
> I have no idea in which logs this would be put from the django-side
>

Not sure what you guys running for a server (gunicorn, apache, etc), but
if you're using systemd you should be able to run something similar to
this (you most likely will need to change the unit for the server you're
using):

`sudo journalctl -u gunicorn --since today`

For example, running this on Metron, I see something like this from
users that are authenticated (and using a non-browser user agent):

```

<snip>

Jul 20 19:59:01 Metron:  - transient-man [20/Jul/2025:15:59:01 -0400]
"GET /api/issue/52077/ HTTP/1.0" 200 3168 "-" "transient/0.1.1"
Jul 20 19:59:02 Metron:  - illum11 [20/Jul/2025:15:59:02 -0400] "GET
/api/issue/?cv_id=797871 HTTP/1.0" 200 377 "-" "ComicBookFetcher/1.0
(+https://thecollectiblespot.com)"
Jul 20 19:59:08 Metron:  - transient-man [20/Jul/2025:15:59:08 -0400]
"GET /api/issue/52078/ HTTP/1.0" 200 2893 "-" "transient/0.1.1"

<snip>

```

You could filter to the request I made earlier by appending a `-g
"Posting/2.7.0"` to the journalctl command from above.

It might be worthwhile to use Postman or Posting to test the your
changes, since it's a fairly simple, and you can view both sides of the
request being made.

Taker care,

[Jochen G.]

needed to change some apache setting, works now, with enhanced
cloudflare as well

Am 20.07.25 um 22:25 schrieb Brian Pepple:

[Brian Pepple]

On 7/20/25 17:11, 'Jochen G.' via gcd-tech wrote:
> needed to change some apache setting, works now, with enhanced
> cloudflare as well
>

Confirmed on my end also. Thanks for all the time you spent working on
this it's much appreciated.

Take,