[GrandComicsDatabase/gcd-django] Sprint 2 api-v2 public endpoints and performance follow-ups (PR #713)

Adam Hernandez

unread,

May 12, 2026, 8:04:23 PM (6 days ago) May 12

to GrandComicsDatabase/gcd-django, Subscribed

Summary

This PR completes the Sprint 2 public api-v2 surface and includes the follow-up performance work needed to keep the new endpoints usable against production-scale data.

Included in this branch:

optimize series browse payloads and add the v2 browse index
add issues.on_sale_iso_week
add public universes endpoints
add public groups endpoints
add public characters endpoints
add public creators endpoints
verify the Sprint 2 public schema and route surface
fix legacy creator partial-date serialization on production data
add browse/delta performance indexes and queryset optimizations for characters, groups, and issues
skip exact counts for issue delta-sync requests to avoid multi-second pagination cost

Verification

Code-level verification:

ruff check passed on changed files
ruff format --check passed on changed files
python manage.py check passed in gcd-django-docker-web-1
focused filter / serializer / view / model / perf pytest slices passed for the new Sprint 2 work

Manual verification against the running production-copy DB:

all Sprint 2 manifests returned 200
public schema and route surface were verified
perf harness runs were repeated using the same Phase 1 methodology

Representative perf improvements from the manual harness:

issues modified__gt=2025-01-01: 2564.6ms -> 48.0ms
issues variant_only: 798.0ms -> 85.5ms
characters list: 213.5ms -> 48.2ms
characters language=en: 181.1ms -> 77.9ms
groups list: 180.0ms -> 98.0ms

You can view, comment on, or merge this pull request online at:

https://github.com/GrandComicsDatabase/gcd-django/pull/713

Commit Summary

084db13 perf(api-v2): slim series browse payloads and add browse index
7a51473 feat(api-v2): add on_sale_iso_week filter for issues
43d1b5e feat(api-v2): add universes endpoints
bfb3c69 feat(api-v2): add groups endpoints
a8317cc feat(api-v2): add characters endpoints
3d1162e feat(api-v2): add creators endpoints
dcce69f test(api-v2): verify sprint-2 schema and public surface
04b6e10 fix(api-v2): handle legacy creator partial dates
88e6f0b perf(api-v2): add character and group browse indexes
622ea28 perf(api-v2): optimize issue modified delta queries
1e644ff perf(api-v2): optimize issue browse and variant queries
ff361d4 perf(api-v2): skip exact counts for issue delta sync
4f1d7fe perf(api-v2): optimize character language filtering
3a76f45 perf(api-v2): trim character browse query footprint

File Changes

(55 files)

A apps/api_v2/filters/characters.py (110)
A apps/api_v2/filters/creators.py (196)
A apps/api_v2/filters/groups.py (79)
M apps/api_v2/filters/issues.py (42)
A apps/api_v2/filters/universes.py (73)
M apps/api_v2/pagination.py (109)
A apps/api_v2/serializers/characters.py (113)
A apps/api_v2/serializers/creators.py (193)
A apps/api_v2/serializers/groups.py (110)
M apps/api_v2/serializers/issues.py (39)
M apps/api_v2/serializers/series.py (23)
A apps/api_v2/serializers/universes.py (47)
M apps/api_v2/tests/conftest.py (9)
A apps/api_v2/tests/test_filters/test_characters.py (384)
A apps/api_v2/tests/test_filters/test_creators.py (278)
A apps/api_v2/tests/test_filters/test_groups.py (259)
M apps/api_v2/tests/test_filters/test_issues.py (73)
A apps/api_v2/tests/test_filters/test_universes.py (196)
A apps/api_v2/tests/test_pagination.py (42)
A apps/api_v2/tests/test_performance/test_characters.py (177)
A apps/api_v2/tests/test_performance/test_creators.py (164)
A apps/api_v2/tests/test_performance/test_groups.py (177)
M apps/api_v2/tests/test_performance/test_issues.py (2)
M apps/api_v2/tests/test_performance/test_series.py (2)
A apps/api_v2/tests/test_performance/test_universes.py (86)
A apps/api_v2/tests/test_serializers/test_characters.py (198)
A apps/api_v2/tests/test_serializers/test_creators.py (300)
A apps/api_v2/tests/test_serializers/test_groups.py (168)
M apps/api_v2/tests/test_serializers/test_issues.py (28)
M apps/api_v2/tests/test_serializers/test_series.py (64)
A apps/api_v2/tests/test_serializers/test_universes.py (89)
M apps/api_v2/tests/test_url_dispatch.py (135)
A apps/api_v2/tests/test_views/test_characters.py (386)
A apps/api_v2/tests/test_views/test_creators.py (381)
A apps/api_v2/tests/test_views/test_groups.py (380)
M apps/api_v2/tests/test_views/test_issues.py (110)
M apps/api_v2/tests/test_views/test_series.py (9)
A apps/api_v2/tests/test_views/test_universes.py (266)
M apps/api_v2/urls_www.py (8)
A apps/api_v2/views/characters.py (123)
A apps/api_v2/views/creators.py (116)
A apps/api_v2/views/groups.py (101)
M apps/api_v2/views/issues.py (125)
M apps/api_v2/views/series.py (99)
A apps/api_v2/views/universes.py (60)
A apps/gcd/migrations/0067_series_api_v2_browse_index.py (20)
A apps/gcd/migrations/0068_character_group_api_v2_browse_indexes.py (39)
A apps/gcd/migrations/0069_issue_api_v2_modified_index.py (20)
A apps/gcd/migrations/0070_issue_api_v2_browse_variant_indexes.py (27)
A apps/gcd/migrations/0071_character_api_v2_language_browse_index.py (27)
M apps/gcd/models/character.py (35)
M apps/gcd/models/issue.py (14)
M apps/gcd/models/series.py (6)
A apps/gcd/tests/test_character.py (55)
M apps/gcd/tests/test_issue.py (33)

Patch Links:

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.

Adam Hernandez

unread,

May 12, 2026, 8:12:18 PM (6 days ago) May 12

to GrandComicsDatabase/gcd-django, Push

@DeusExTaco pushed 1 commit.

fab2d29 test(api-v2): use testserver in issue pagination unit test

—
View it on GitHub or unsubscribe.

Triage notifications on the go with GitHub Mobile for iOS or Android.

You are receiving this because you are subscribed to this thread.

gemini-code-assist[bot]

unread,

May 12, 2026, 8:12:28 PM (6 days ago) May 12

to GrandComicsDatabase/gcd-django, Subscribed

@gemini-code-assist[bot] commented on this pull request.

Code Review

This pull request adds new v2 API endpoints for characters, creators, groups, and universes, and optimizes existing issue and series endpoints with specialized pagination and database indexing. The review feedback recommends enhancing robustness against non-numeric legacy data in integer casts, using .get() instead of .first() for unique lookups to ensure data integrity, and utilizing queryset intersections for more efficient issue filtering.

In apps/api_v2/filters/creators.py:

> +        When(**{f'{field_name}__isnull': True}, then=Value(0)),
+        When(**{f'{field_name}__{component}': ''}, then=Value(0)),
+        default=F(f'{field_name}__{component}'),
+        output_field=IntegerField(),

Using F() with output_field=IntegerField() on CharField fields like year, month, and day can cause runtime errors if the database contains non-numeric strings (e.g., legacy markers like '????' or '1940?'). While MySQL might implicitly cast these to an integer (e.g., '1940?' to 1940), other databases like PostgreSQL will raise an error. Consider explicitly handling non-numeric values in the When clauses to ensure cross-database compatibility and robustness against legacy data.

In apps/api_v2/filters/characters.py:

> +            language_id = (
+                Language.objects.filter(code=value)
+                .values_list('id', flat=True)
+                .first()
+            )

As per the general rules, prefer using .get() instead of .first() when the data model expects a single object. This helps identify data integrity issues (like duplicate language codes) by raising a MultipleObjectsReturned exception. You will need to handle the DoesNotExist exception to maintain the current logic for invalid codes.

            try:
                language_id = Language.objects.values_list('id', flat=True).get(code=value)
            except Language.DoesNotExist:
                language_id = None

References

Prefer using .get() to raise a MultipleObjectsReturned exception when the data model expects a single object. This helps to identify and fix data integrity issues, rather than silently picking the first object with .first().

In apps/api_v2/filters/issues.py:

> @@ -110,3 +142,13 @@ def filter_variant_of(self, queryset, name, value):
         """Filter by whether an issue is a variant."""
         del name
         return queryset.filter(variant_of__isnull=not value)
+
+    def filter_on_sale_iso_week(self, queryset, name, value):
+        """Filter issues to the Monday-Sunday window for an ISO week."""
+        del name
+        year_text, week_text = value.split('-W')
+        week_qs, _monday, _sunday = issues_for_iso_week(
+            int(year_text),
+            int(week_text),
+        )
+        return queryset.filter(pk__in=week_qs.values('pk'))

Since both queryset and week_qs are querysets for the Issue model, you can use the intersection operator & to combine them. This is generally more efficient than a pk__in subquery as it allows the database to optimize the combined WHERE clause directly, which aligns with the performance goals of this PR.

⬇️ Suggested change

-        return queryset.filter(pk__in=week_qs.values('pk'))
+        return queryset & week_qs

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

You are receiving this because you are subscribed to this thread.

Adam Hernandez

unread,

May 12, 2026, 8:35:01 PM (6 days ago) May 12

to GrandComicsDatabase/gcd-django, Subscribed

@DeusExTaco commented on this pull request.

In apps/api_v2/filters/issues.py:

> @@ -110,3 +142,13 @@ def filter_variant_of(self, queryset, name, value):
         """Filter by whether an issue is a variant."""
         del name
         return queryset.filter(variant_of__isnull=not value)
+
+    def filter_on_sale_iso_week(self, queryset, name, value):
+        """Filter issues to the Monday-Sunday window for an ISO week."""
+        del name
+        year_text, week_text = value.split('-W')
+        week_qs, _monday, _sunday = issues_for_iso_week(
+            int(year_text),
+            int(week_text),
+        )
+        return queryset.filter(pk__in=week_qs.values('pk'))

Roger that...I switched this over to queryset intersection instead of the pk__in subquery.

That keeps the generated SQL as a direct on_sale_date predicate, including the split month-
boundary week case, which is a better fit for the perf work in this PR. I also added a small
regression test around the SQL shape.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

You are receiving this because you are subscribed to this thread.

Adam Hernandez

unread,

May 12, 2026, 8:35:53 PM (6 days ago) May 12

to GrandComicsDatabase/gcd-django, Subscribed

@DeusExTaco commented on this pull request.

In apps/api_v2/filters/characters.py:

> +            language_id = (
+                Language.objects.filter(code=value)
+                .values_list('id', flat=True)
+                .first()
+            )

I’m going to leave this one as-is.

Language.code is already unique in the schema, so .get() doesn’t really buy us anything in the
normal case here. The behavior I want on bad query params is still “no matches” rather than an
exception path, and the current code keeps that nice and soft while also letting us cache the
resolved language_id on the request.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

You are receiving this because you are subscribed to this thread.

Adam Hernandez

unread,

May 12, 2026, 8:50:21 PM (6 days ago) May 12

to GrandComicsDatabase/gcd-django, Push

@DeusExTaco pushed 2 commits.

67155fe perf(api-v2): inline issue iso-week filter predicates
7173db1 fix(api-v2): ignore uncertain creator dates in range filters

—
View it on GitHub or unsubscribe.

Triage notifications on the go with GitHub Mobile for iOS or Android.

You are receiving this because you are subscribed to this thread.

Adam Hernandez

unread,

May 12, 2026, 8:51:20 PM (6 days ago) May 12

to GrandComicsDatabase/gcd-django, Subscribed

@DeusExTaco commented on this pull request.

In apps/api_v2/filters/creators.py:

> +        When(**{f'{field_name}__isnull': True}, then=Value(0)),
+        When(**{f'{field_name}__{component}': ''}, then=Value(0)),
+        default=F(f'{field_name}__{component}'),
+        output_field=IntegerField(),

This was an interesting one....

The current filter logic was relying on implicit integer coercion from the stored Date char
fields, which gets weird with legacy values like ????, 19??, and 200?. I changed the range
filters so they only build sort keys from numeric stored components and ignore non-comparable
legacy partial dates instead of trying to coerce them.

That keeps the behavior consistent for birth_date__gte/lte and death_date__gte/lte, and avoids
those uncertain rows leaking into lte results.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

You are receiving this because you are subscribed to this thread.

Adam Hernandez

unread,

May 17, 2026, 3:41:14 PM (2 days ago) May 17

to gcd-tech

Just bumping this so someone can take a look at my PR please. Thanks!

Adam

JochenGCD

unread,

May 18, 2026, 12:47:39 PM (14 hours ago) May 18

to GrandComicsDatabase/gcd-django, Subscribed

@jochengcd commented on this pull request.

In apps/api_v2/filters/creators.py:

> +        When(**{f'{field_name}__isnull': True}, then=Value(0)),
+        When(**{f'{field_name}__{component}': ''}, then=Value(0)),
+        default=F(f'{field_name}__{component}'),
+        output_field=IntegerField(),

These are not legacy values, but valid data in our database. If we know only partial information we fill these with ?. So 200? means sometime between 2000 and 2009.

For filtering we use these in the site.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

You are receiving this because you are subscribed to this thread.

JochenGCD

unread,

May 18, 2026, 12:52:12 PM (14 hours ago) May 18

to GrandComicsDatabase/gcd-django, Subscribed

@jochengcd commented on this pull request.

In apps/api_v2/filters/issues.py:

> @@ -110,3 +142,13 @@ def filter_variant_of(self, queryset, name, value):
         """Filter by whether an issue is a variant."""
         del name
         return queryset.filter(variant_of__isnull=not value)
+
+    def filter_on_sale_iso_week(self, queryset, name, value):
+        """Filter issues to the Monday-Sunday window for an ISO week."""
+        del name
+        year_text, week_text = value.split('-W')
+        week_qs, _monday, _sunday = issues_for_iso_week(
+            int(year_text),
+            int(week_text),
+        )
+        return queryset.filter(pk__in=week_qs.values('pk'))

More out of curiosity, did you do time measurements here ?

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

You are receiving this because you are subscribed to this thread.

Adam Hernandez

unread,

May 18, 2026, 3:37:51 PM (11 hours ago) May 18

to GrandComicsDatabase/gcd-django, Subscribed

@DeusExTaco commented on this pull request.

In apps/api_v2/filters/issues.py:

> @@ -110,3 +142,13 @@ def filter_variant_of(self, queryset, name, value):
         """Filter by whether an issue is a variant."""
         del name
         return queryset.filter(variant_of__isnull=not value)
+
+    def filter_on_sale_iso_week(self, queryset, name, value):
+        """Filter issues to the Monday-Sunday window for an ISO week."""
+        del name
+        year_text, week_text = value.split('-W')
+        week_qs, _monday, _sunday = issues_for_iso_week(
+            int(year_text),
+            int(week_text),
+        )
+        return queryset.filter(pk__in=week_qs.values('pk'))

I went back and measured this on the local production-copy DB instead of just relying on the SQL shape. I compared the old pk__in version with the current queryset intersection, using 3 warmup runs and 12 measured runs for each, and the intersection version was consistently faster, so I think this is still the right change to keep.

2025-W13 (575 matches): count() 0.739ms -> 0.477ms, first page 5.337ms -> 3.025ms
2025-W05 (399 matches, split-month week): count() 0.620ms -> 0.444ms, first page 4.693ms -> 3.286ms
Broader PR context from the same production-copy perf pass: issues modified__gt=2025-01-01 2564.6ms -> 48.0ms
Broader PR context: issues variant_only 798.0ms -> 85.5ms

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

You are receiving this because you are subscribed to this thread.

Adam Hernandez

unread,

May 18, 2026, 3:50:18 PM (11 hours ago) May 18

to GrandComicsDatabase/gcd-django, Subscribed

@DeusExTaco commented on this pull request.

In apps/api_v2/filters/creators.py:

> +        When(**{f'{field_name}__isnull': True}, then=Value(0)),
+        When(**{f'{field_name}__{component}': ''}, then=Value(0)),
+        default=F(f'{field_name}__{component}'),
+        output_field=IntegerField(),

Good catch here. I had made the filter too strict while fixing the cast issue. I’ve updated the v2 creator date filtering so partial years like 19?? and 200? are treated as bounded ranges again instead of getting dropped, while fully unknown values like ???? still stay out of date-range matches. I also added coverage for the filter and endpoint behavior and spot-checked it against the local production-copy DB.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

You are receiving this because you are subscribed to this thread.

Adam Hernandez

unread,

May 18, 2026, 3:54:01 PM (11 hours ago) May 18

to GrandComicsDatabase/gcd-django, Push

@DeusExTaco pushed 1 commit.

8f876e9 fix(api-v2): support uncertain creator date filters

—
View it on GitHub or unsubscribe.

Triage notifications on the go with GitHub Mobile for iOS or Android.

You are receiving this because you are subscribed to this thread.

Reply all

Reply to author

Forward