beta head (3127822a) into api-v2 after the Phase 3 mergebeta.dockerignore so local metadata, caches, bytecode, and settings_local.py stay out of Docker build contextsapi-v2 currently ends at the Phase 3 merge (fd7b1531), while Phase 4 was developed and validated on top of the current beta model state. Landing this synchronization first keeps the Phase 4 pull request limited to its seven feature commits instead of mixing them with the intervening beta history.
This is a prerequisite synchronization PR. It does not add the Phase 4 endpoints. Once merged, the Phase 4 branch can target api-v2 with only the Indicia Publisher, Indicia Printer, Brand Group, Brand, Feature, Award, and Series Bond work in its review diff.
ruff check apps/api_v2/ruff format --check apps/api_v2/python manage.py checkpython -m pytest apps/api_v2/tests/ -q --tb=short (203 passed, 24 existing warnings)The descendant Phase 4 branch has also passed its complete 313-test API v2 suite and production-copy manual baseline.
https://github.com/GrandComicsDatabase/gcd-django/pull/729
(60 files)
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.![]()
@gemini-code-assist[bot] commented on this pull request.
This pull request introduces a character ordering feature (by appearance or importance) with corresponding database models, forms, views, and templates, alongside a new API endpoint for series overviews. The code review highlights a critical bug in CharacterOrderRevision._post_save_object where an AttributeError would be raised, and a potential ValueError in StoryRevisionForm on unsaved instances. Additionally, several N+1 query bottlenecks and database query inefficiencies were identified in character processing and reordering logic, along with minor code smells regarding type checking and class attribute placeholders.
> + def _post_save_object(self, changes): + characters = self.character_order.characters.all() + character_revisions = self.character_revisions.all() + for character in characters: + if not character_revisions.filter( + character__id=character.id, + universe=character.universe).count(): + self.character_order.characters.remove(character) + else: + character.order_code = character_revisions.get( + character__id=character.id, + universe=character.universe).order_code + character.save() + for character_revision in character_revisions: + if not characters.filter( + id=character_revision.character.id, + universe=character_revision.universe).exists(): + order_code = character_revision\ + .characterthroughorderrevision_set.get(order=self).order_code + self.character_order.characters.add( + character_revision.story_character, + through_defaults={'order_code': order_code})
This method contains a critical bug: character_revisions.get(...).order_code will raise an AttributeError because StoryCharacterRevision does not have an order_code attribute (it is defined on the through model CharacterThroughOrderRevision). Additionally, character.save() fails to update the through table, and both loops perform N+1 queries. This optimized implementation fixes the bugs and eliminates all database queries inside the loops.
def _post_save_object(self, changes): characters = self.character_order.characters.all() character_revisions = self.character_revisions.all() through_instances = { ti.story_character_id: ti for ti in self.character_order.characterthroughorder_set.all() } revision_through_instances = { rti.story_character_id: rti.order_code for rti in self.character_revisions.through.objects.filter(order=self) } for character in characters: matching_revision = character_revisions.filter( character__id=character.id, universe=character.universe ).first() if not matching_revision: self.character_order.characters.remove(character) else: through_instance = through_instances.get(character.id) new_order_code = revision_through_instances.get(matching_revision.id, 0) if through_instance and through_instance.order_code != new_order_code: through_instance.order_code = new_order_code through_instance.save(update_fields=['order_code']) for character_revision in character_revisions: if not characters.filter( id=character_revision.character.id, universe=character_revision.universe).exists(): order_code = revision_through_instances.get(character_revision.id, 0) self.character_order.characters.add( character_revision.story_character, through_defaults={'order_code': order_code})
> + if instance.characterorderrevision_set.exists(): + self.no_delete = True + self.fields['character'].help_text = \ + 'Characters that are part of a character order cannot be '\ + 'removed.'
Accessing instance.characterorderrevision_set on an unsaved model instance (where pk is None) will raise a ValueError in Django. We should guard this check by ensuring instance.pk is present.
- if instance.characterorderrevision_set.exists(): - self.no_delete = True - self.fields['character'].help_text = \ - 'Characters that are part of a character order cannot be '\ - 'removed.' + if instance.pk and instance.characterorderrevision_set.exists(): + self.no_delete = True + self.fields['character'].help_text = \ + 'Characters that are part of a character order cannot be '\\ + 'removed.'
In apps/gcd/models/creator.py:
> + if as_name != self.name: + if as_name.creator.active_names().filter( + name=self.name): + as_name = as_name.creator.active_names().filter( + name=self.name)[0]
The query as_name.creator.active_names().filter(name=self.name) is executed twice in a row: first to check if it exists, and then to retrieve the first element. This can be optimized to a single query using .first() to improve database efficiency.
if as_name != self.name: matching_name = as_name.creator.active_names().filter(name=self.name).first() if matching_name: as_name = matching_name
> +def process_ordered_appearing_characters(character_order):
+ """
+ Return a properly formatted list of characters appearing in the story.
+ The order ist defined by the given CharacterOrder, followed by any other
+ appearing characters not included in the order, ordered by sort name.
+ """
+ story = character_order.story
+ all_appearing_characters = story.active_characters
+ in_group = all_appearing_characters.exclude(group_name=None)
+ if hasattr(character_order, 'character_revisions'):
+ field = 'character_revisions'
+ else:
+ field = 'characters'
+ through_model = character_order._meta.get_field(field).remote_field.through
+ in_character_order = all_appearing_characters.filter(
+ **{f'{through_model.__name__.lower()}__order': character_order}
+ ).distinct()
+ groups = story.active_groups
+
+ reference_universe_id = _get_reference_universe(story)
+
+ group_list = []
+ processed_appearances_ids = []
+ for group in groups:
+ group_universe = None
+ if reference_universe_id and group.universe:
+ if group.universe_id != reference_universe_id:
+ group_universe = group.universe
+ character_list = []
+ ordered_character_list = []
+ for member in in_group.filter(group_name=group.group_name_id,
+ group_universe=group.universe_id):
+ if member in in_character_order:
+ ordered_character_list.append((
+ getattr(character_order,
+ f'{through_model.__name__.lower()}_set').get(
+ order=character_order,
+ story_character=member).order_code, member))
+ else:
+ character_list.append(_process_single_character(
+ member, all_appearing_characters, reference_universe_id))
+ processed_appearances_ids.append(member.id)
+ ordered_character_list.sort(key=lambda x: x[0])
+ cnt = 0
+ for _, member in ordered_character_list:
+ character_list.insert(cnt, _process_single_character(
+ member, all_appearing_characters, reference_universe_id))
+ cnt += 1
+ group_list.append((group, group_universe, character_list))
+ appearing_characters = all_appearing_characters.exclude(
+ id__in=processed_appearances_ids)
+
+ character_list = []
+ ordered_character_list = []
+ for character in appearing_characters:
+ alias_identity = set(
+ character.character.character.from_related_character
+ .filter(relation_type__id=2).values_list('from_character',
+ flat=True))\
+ .intersection(all_appearing_characters.filter(
+ universe=character.universe).values_list(
+ 'character__character', flat=True))
+ if alias_identity:
+ continue
+ if character in in_character_order:
+ ordered_character_list.append((
+ getattr(character_order,
+ f'{through_model.__name__.lower()}_set').get(
+ order=character_order,
+ story_character=character).order_code, character))
+ else:
+ character_list.append(_process_single_character(
+ character, all_appearing_characters, reference_universe_id))
+ ordered_character_list.sort(key=lambda x: x[0])
+ cnt = 0
+ for _, character in ordered_character_list:
+ character_list.insert(cnt, _process_single_character(
+ character, all_appearing_characters, reference_universe_id))
+ cnt += 1
+ return (group_list, character_list)
The process_ordered_appearing_characters function performs multiple database queries inside loops (N+1 query pattern) by calling .get() on the through model for each character. We can optimize this by pre-fetching all through model instances in bulk and mapping them by character ID.
def process_ordered_appearing_characters(character_order): """ Return a properly formatted list of characters appearing in the story. The order is defined by the given CharacterOrder, followed by any other appearing characters not included in the order, ordered by sort name. """ story = character_order.story all_appearing_characters = story.active_characters in_group = all_appearing_characters.exclude(group_name=None) if hasattr(character_order, 'character_revisions'): field = 'character_revisions' else: field = 'characters' through_model = character_order._meta.get_field(field).remote_field.through in_character_order = all_appearing_characters.filter( **{f'{through_model.__name__.lower()}__order': character_order} ).distinct() groups = story.active_groups reference_universe_id = _get_reference_universe(story) through_relation = getattr(character_order, f'{through_model.__name__.lower()}_set') order_map = { instance.story_character_id: instance.order_code for instance in through_relation.all() } group_list = [] processed_appearances_ids = [] for group in groups: group_universe = None if reference_universe_id and group.universe: if group.universe_id != reference_universe_id: group_universe = group.universe character_list = [] ordered_character_list = [] for member in in_group.filter(group_name=group.group_name_id, group_universe=group.universe_id): if member in in_character_order: ordered_character_list.append((order_map.get(member.id, 0), member)) else: character_list.append(_process_single_character( member, all_appearing_characters, reference_universe_id)) processed_appearances_ids.append(member.id) ordered_character_list.sort(key=lambda x: x[0]) cnt = 0 for _, member in ordered_character_list: character_list.insert(cnt, _process_single_character( member, all_appearing_characters, reference_universe_id)) cnt += 1 group_list.append((group, group_universe, character_list)) appearing_characters = all_appearing_characters.exclude( id__in=processed_appearances_ids) character_list = [] ordered_character_list = [] for character in appearing_characters: alias_identity = set( character.character.character.from_related_character .filter(relation_type__id=2).values_list('from_character', flat=True))\\ .intersection(all_appearing_characters.filter( universe=character.universe).values_list( 'character__character', flat=True)) if alias_identity: continue if character in in_character_order: ordered_character_list.append((order_map.get(character.id, 0), character)) else: character_list.append(_process_single_character( character, all_appearing_characters, reference_universe_id)) ordered_character_list.sort(key=lambda x: x[0]) cnt = 0 for _, character in ordered_character_list: character_list.insert(cnt, _process_single_character( character, all_appearing_characters, reference_universe_id)) cnt += 1 return (group_list, character_list)
> + # H.TODO source_name = NotImplementedError + source_class = NotImplementedError
Setting a class attribute placeholder to NotImplementedError (the exception class itself) is a code smell in Python. It is cleaner and more idiomatic to use None as a placeholder for class attributes that must be overridden by subclasses.
- # H.TODO source_name = NotImplementedError - source_class = NotImplementedError + # H.TODO source_name = NotImplementedError + source_class = None
In apps/oi/views.py:
> + order = 0 + for character in characters: + revision_characters = character_order_revision.character_revisions + if not revision_characters.filter(id=character.id).exists(): + revision_characters.add(character, + through_defaults={'order_code': order}) + else: + through_instance = revision_characters.through.objects.get( + order=character_order_revision, + story_character=character + ) + through_instance.order_code = order + through_instance.save() + order += 1
This loop performs a database query (.get()) and a .save() for each character, leading to an N+1 query bottleneck. We can optimize this by pre-fetching the existing through instances and only calling .save() if the order_code has actually changed.
- order = 0 - for character in characters: - revision_characters = character_order_revision.character_revisions - if not revision_characters.filter(id=character.id).exists(): - revision_characters.add(character, - through_defaults={'order_code': order}) - else: - through_instance = revision_characters.through.objects.get( - order=character_order_revision, - story_character=character - ) - through_instance.order_code = order - through_instance.save() - order += 1 + through_instances = { + ti.story_character_id: ti + for ti in character_order_revision.character_revisions.through.objects.filter( + order=character_order_revision + ) + } + revision_characters = character_order_revision.character_revisions + for order, character in enumerate(characters): + through_instance = through_instances.get(character.id) + if not through_instance: + revision_characters.add(character, + through_defaults={'order_code': order}) + elif through_instance.order_code != order: + through_instance.order_code = order + through_instance.save(update_fields=['order_code'])
In apps/oi/views.py:
> @@ -601,6 +605,21 @@ def submit(request, id):
if comment_text:
send_comment_observer(request, changeset, comment_text)
+ # If there are only CharacterOrderRevisions, and no actual changes,
+ # we can skip the reviewing and commit.
+ if changeset.imps == 0 and changeset.characterorderrevisions.exists():
+ is_changed = False
+ for c in changeset.revisions:
+ c.compare_changes()
+ if c.is_changed and type(c) is not CharacterOrderRevision:
Using type(c) is not CharacterOrderRevision is less idiomatic in Python. It is recommended to use not isinstance(c, CharacterOrderRevision) to correctly support potential subclasses and adhere to standard Python style guidelines.
- if c.is_changed and type(c) is not CharacterOrderRevision: + if c.is_changed and not isinstance(c, CharacterOrderRevision):
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.![]()
@jochengcd commented on this pull request.
> + if instance.characterorderrevision_set.exists(): + self.no_delete = True + self.fields['character'].help_text = \ + 'Characters that are part of a character order cannot be '\ + 'removed.'
there is an if instance just befire
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.![]()
@DeusExTaco commented on this pull request.
In apps/oi/views.py:
> + order = 0 + for character in characters: + revision_characters = character_order_revision.character_revisions + if not revision_characters.filter(id=character.id).exists(): + revision_characters.add(character, + through_defaults={'order_code': order}) + else: + through_instance = revision_characters.through.objects.get( + order=character_order_revision, + story_character=character + ) + through_instance.order_code = order + through_instance.save() + order += 1
Confirmed as the other inherited character-order N+1 path. It is tracked with the display-side concern in #731. Keeping this synchronization PR unchanged and closing this thread as out of scope here.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.![]()
@DeusExTaco commented on this pull request.
In apps/gcd/models/creator.py:
> + if as_name != self.name: + if as_name.creator.active_names().filter( + name=self.name): + as_name = as_name.creator.active_names().filter( + name=self.name)[0]
Confirmed as a minor query optimization in code already present on beta. It is tracked separately in #732 so #729 does not diverge from the beta state it is intended to synchronize. Closing as out of scope for this PR.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.![]()
@DeusExTaco commented on this pull request.
Confirmed as an inherited N+1 concern in current beta. The display and editing paths are tracked together in #731 with query-count regression coverage. Keeping this synchronization PR unchanged and closing this thread as out of scope here.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.![]()
@DeusExTaco commented on this pull request.
> + def _post_save_object(self, changes): + characters = self.character_order.characters.all() + character_revisions = self.character_revisions.all() + for character in characters: + if not character_revisions.filter( + character__id=character.id, + universe=character.universe).count(): + self.character_order.characters.remove(character) + else: + character.order_code = character_revisions.get( + character__id=character.id, + universe=character.universe).order_code + character.save() + for character_revision in character_revisions: + if not characters.filter( + id=character_revision.character.id, + universe=character_revision.universe).exists(): + order_code = character_revision\ + .characterthroughorderrevision_set.get(order=self).order_code + self.character_order.characters.add( + character_revision.story_character, + through_defaults={'order_code': order_code})
Confirmed: this is a real correctness issue in code already present on beta, rather than a defect introduced by this synchronization PR. A complete fix needs to update the through rows and include regression coverage, so it is tracked separately in #730. I am keeping #729 as an exact beta-to-api-v2 sync and closing this thread as out of scope here.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.![]()
@DeusExTaco commented on this pull request.
In apps/oi/views.py:
> @@ -601,6 +605,21 @@ def submit(request, id):
if comment_text:
send_comment_observer(request, changeset, comment_text)
+ # If there are only CharacterOrderRevisions, and no actual changes,
+ # we can skip the reviewing and commit.
+ if changeset.imps == 0 and changeset.characterorderrevisions.exists():
+ is_changed = False
+ for c in changeset.revisions:
+ c.compare_changes()
+ if c.is_changed and type(c) is not CharacterOrderRevision:
While isinstance() is normally more idiomatic, this exact-type check is existing beta behavior and changing it could alter how future subclasses participate in auto-approval. There is no demonstrated defect to fix as part of this synchronization PR, so this remains unchanged and the thread is being closed as out of scope.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.![]()
Review follow-up:
No code changes were made to #729. All review threads now have a recorded disposition and are resolved.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.![]()
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.![]()