[GrandComicsDatabase/gcd-django] replacing Images (Issue #663)
JochenGCD
jochengcd created an issue (GrandComicsDatabase/gcd-django#663)When replacing images, the cache at Cloudflare keeps these for a bit longer, be it the full file, be it the processed versions via imagekit in the image-cache.
We could purge the Cloudflare cache for these occasions, if it becomes an issue noticed by users, e.g. when updating Creator images, or brand emblems.
https://developers.cloudflare.com/api/python/resources/cache/methods/purge/
import os
from cloudflare import Cloudflare
client = Cloudflare(
api_email=os.environ.get("CLOUDFLARE_EMAIL"), # This is the default and can be omitted
api_key=os.environ.get("CLOUDFLARE_API_KEY"), # This is the default and can be omitted
)
response = client.cache.purge(
zone_id="zone_id",
)
print(response.id)
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
Adam Hernandez
DeusExTaco left a comment (GrandComicsDatabase/gcd-django#663)Looking at the current image replacement flow, and I believe we can address this without adding Cloudflare API calls as the first step. GCD already tries to prevent stale images by adding a changing query string to some image URLs. That is generally the right approach because it gives Cloudflare and the browser a new URL whenever an image changes.
The current implementation has a couple of gaps though. The versioned URL is only used in certain image helpers, while a number of creator portraits, brand emblems, feature logos, thumbnails, icons, and original-image links still use the unversioned URL directly. The version is also generated with Python’s hash() function, which can produce different values in different web-server processes. That means two workers can generate different URLs for the same unchanged image, making the cache less predictable and less efficient.
I would suggest to add one shared, deterministic image-version helper based on the existing modified or last_upload timestamp, and use it everywhere an approved image or ImageKit derivative is displayed. We should also make sure all relevant ImageKit files are cleared when a replacement is approved. Tests can then confirm that an unchanged image always gets the same URL, approving a replacement changes it, and the creator, brand, feature, issue-image, and cover pages all use the versioned URLs. This shouldn't require a database migration.
I’d keep targeted Cloudflare purging as a fallback rather than making it part of this first option. Adding the API would introduce credentials, another dependency, and network-failure handling into the image approval process. If versioned URLs don’t fully solve the problem, we could add a narrowly scoped Cache Purge API token later and purge only the affected files rather than the entire zone.
A couple of questions...
- Does the Cloudflare configuration for images.comics.org use the default cache key that includes query strings, or is there a rule that ignores them?
- Can you provide a recent stale image URL and let me know whether it was a creator portrait, brand emblem, feature logo, issue image, or cover? That would let me reproduce the exact path and verify the fix.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.
JochenGCD
jochengcd left a comment (GrandComicsDatabase/gcd-django#663)Covers are treated differently then other images, these we treat 'by hand', no imagekit at that time. There we add the query string.
All other images are via imagekit and there we have the cache issue, i.e. when these are (rarely) replaced. One can monitor the images section in the pending queue for a replacement upload.
Looking in the db for one won't be helpful. The image is stale for a some hours only.
I don't think we have anything specific at cloudflare for files.comics.org
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.
Adam Hernandez
DeusExTaco left a comment (GrandComicsDatabase/gcd-django#663)Thanks, that clears up the scope. I’ll leave covers out of this since they’re handled separately and already use query-string versioning. I’ll focus on the ImageKit-backed images and use the images section of the pending queue to catch a replacement so I can compare the original and generated URLs before and after approval.
If there isn’t a custom Cloudflare cache rule for files.comics.org, the default behavior should treat a changed query string as a new cache key. That makes consistent cache versioning the simplest first fix. I’ll replace the current process-dependent hash() value with a deterministic value based on modified, add that version to every original and ImageKit-generated URL instead of only selected helpers, and confirm that all derivative cache files are cleared when a replacement is approved.
I’ll also add tests confirming that an unchanged image keeps the same URL and an approved replacement gets a new one. If a real replacement is still stale after that, I can look at targeted Cloudflare purging as a follow-up instead of adding API credentials and failure handling now. Does that sound like the right scope?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.
Adam Hernandez
DeusExTaco left a comment (GrandComicsDatabase/gcd-django#663)Also if you approve, then please assign this one to me as well.
Thanks
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.