Re: [Django] #36997: test_double_replaced_migrations_are_checked_correctly fails intermittently under Windows Subsystem for Linux (WSL)

[Django]

#36997: test_double_replaced_migrations_are_checked_correctly fails intermittently
under Windows Subsystem for Linux (WSL)
----------------------------------+--------------------------------------
Reporter: Stewart Matheson | Owner: (none)
Type: Bug | Status: closed
Component: Migrations | Version: dev
Severity: Normal | Resolution: needsinfo
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+--------------------------------------
Comment (by Stewart Matheson):

The issue relates is a file system a race condition that is most likely
being triggered by the unpredictable timings of the filesystem running
under HyperV and WSL. After the test writes the first migration file to
the /tmp folder there is a chance that the filesystem mtime update is
still waiting in the FS journal. When the migration system performs a
second read its reading the filesystem before the mtime has been updated
from the previous write. Because of this timing python's file cache is not
invalidated when the test expects it to be. The result is that the first
migration will be over written and the test will fail when it attempts to
squash migrations that don't exist because they where never written in the
first place.

Running the same test on bare metal linux, docker linux or macos is
unlikely to trigger this issue is their filesystems have timings that are
more predictable than WSL.

Running
{{{
importlib.invalidate_caches()
}}}

Each time after a migration file is written resolves this issue for me as
python will bypass reading the mtime. Given all that I'm not too sure how
you would reproduce this as I would have thought other WSL users would be
running in to this issue.
--
Ticket URL: <https://code.djangoproject.com/ticket/36997#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36997: test_double_replaced_migrations_are_checked_correctly fails intermittently
under Windows Subsystem for Linux (WSL)
----------------------------------+--------------------------------------
Reporter: Stewart Matheson | Owner: (none)

Type: Bug | Status: new

Component: Migrations | Version: dev
Severity: Normal | Resolution:

Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+--------------------------------------

Changes (by Stewart Matheson):

* resolution: needsinfo =>
* status: closed => new

Comment:

Hi Tim,

I provided my investigation and reopened this ticket as you have
suggested.
--
Ticket URL: <https://code.djangoproject.com/ticket/36997#comment:3>

[Django]

#36997: test_double_replaced_migrations_are_checked_correctly fails intermittently
under Windows Subsystem for Linux (WSL)
----------------------------------+--------------------------------------
Reporter: Stewart Matheson | Owner: (none)
Type: Bug | Status: new
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+--------------------------------------

Changes (by gghez):

* cc: gghez (added)
* has_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/36997#comment:4>

[Django]

#36997: test_double_replaced_migrations_are_checked_correctly fails intermittently
under Windows Subsystem for Linux (WSL)
----------------------------------+--------------------------------------
Reporter: Stewart Matheson | Owner: (none)
Type: Bug | Status: new
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+--------------------------------------

Comment (by gghez):

Hello, i got this PR: https://github.com/django/django/pull/21019
--
Ticket URL: <https://code.djangoproject.com/ticket/36997#comment:5>

[Django]

#36997: test_double_replaced_migrations_are_checked_correctly fails intermittently
under Windows Subsystem for Linux (WSL)
----------------------------------+--------------------------------------
Reporter: Stewart Matheson | Owner: (none)
Type: Bug | Status: new
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+--------------------------------------

Comment (by Simon Charette):

I closed the PR as no one has been able to reproduce on WSL so far and
thus the ticket hasn't been accepted; submitting a solution without a
clear demonstration that Django is at fault is precipitated.
--
Ticket URL: <https://code.djangoproject.com/ticket/36997#comment:6>

[Django]

#36997: test_double_replaced_migrations_are_checked_correctly fails intermittently
under Windows Subsystem for Linux (WSL)
----------------------------------+--------------------------------------
Reporter: Stewart Matheson | Owner: (none)

Type: Bug | Status: closed
Component: Migrations | Version: dev
Severity: Normal | Resolution: needsinfo

Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+--------------------------------------

Changes (by Natalia Bidart):

* resolution: => needsinfo
* status: new => closed

Comment:

Hello Stewart, thank you for the detailed investigation.

Based on the current debugging information (related to filesystem timing
and mtime visibility under WSL), and considering that:

1. the issue has not been reproduced by others,
2. there is no clear indication that Django is violating expected
guarantees on supported platforms,
3. adding explicit cache invalidation in Django would also need stronger
justification, as it would introduce overhead and compensate for
environment-specific behavior,

I'll keep this in `needsinfo` as originally triaged.

To move this forward, we would need a reliably reproducible case or
confirmation from multiple environments demonstrating that Django is at
fault. If you are able to provide a minimal reliable reproducer or
additional evidence that this occurs outside of WSL-specific conditions,
we can revisit.
--
Ticket URL: <https://code.djangoproject.com/ticket/36997#comment:7>

[Django]

#36997: test_double_replaced_migrations_are_checked_correctly fails intermittently
under Windows Subsystem for Linux (WSL)
----------------------------------+--------------------------------------
Reporter: Stewart Matheson | Owner: (none)
Type: Bug | Status: closed
Component: Migrations | Version: dev
Severity: Normal | Resolution: needsinfo
Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+--------------------------------------

Comment (by mweinelt):

We're seeing this in nixpkgs specifically on aarch64-linux after upgrading
from 6.0.3 to 6.0.4 on Python 3.14.3. I can reproduce this across various
host machines, but not on Python 3.13.12.


{{{
======================================================================
ERROR: test_double_replaced_migrations_are_checked_correctly
(migrations.test_commands.SquashMigrationsTests.test_double_replaced_migrations_are_checked_correctly)
If replaced migrations are already applied and replacing migrations
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/nix/store/s33vdxkahhrdbyrqyw8yn71kzr44l6a0-python3-3.14.3/lib/python3.14/unittest/case.py",
line 58, in testPartExecutor
yield
File
"/nix/store/s33vdxkahhrdbyrqyw8yn71kzr44l6a0-python3-3.14.3/lib/python3.14/unittest/case.py",
line 669, in run
self._callTestMethod(testMethod)

File
"/nix/store/s33vdxkahhrdbyrqyw8yn71kzr44l6a0-python3-3.14.3/lib/python3.14/unittest/case.py",
line 615, in _callTestMethod
result = method()
^^^^^^^^^^^^^^^
File "/build/source/tests/migrations/test_commands.py", line 3197, in
test_double_replaced_migrations_are_checked_correctly
call_command("migrate", "migrations", interactive=False, verbosity=0)
^^^^^^^^^^^
File "/nix/store/2z5nsfr1b1hrk47as0sa3gxa2mdy2sck-
python3.14-django-6.0.4/lib/python3.14/site-
packages/django/core/management/__init__.py", line 195, in call_command
return command.execute(*args, **defaults)
^^^^^^^^^^^^^^^^^
File "/nix/store/2z5nsfr1b1hrk47as0sa3gxa2mdy2sck-
python3.14-django-6.0.4/lib/python3.14/site-
packages/django/core/management/base.py", line 464, in execute
output = self.handle(*args, **options)
^^^^^^^^^^^^^^^
File "/nix/store/2z5nsfr1b1hrk47as0sa3gxa2mdy2sck-
python3.14-django-6.0.4/lib/python3.14/site-
packages/django/core/management/base.py", line 111, in wrapper
res = handle_func(*args, **kwargs)
^^^^^^^^^^^
File "/nix/store/2z5nsfr1b1hrk47as0sa3gxa2mdy2sck-
python3.14-django-6.0.4/lib/python3.14/site-
packages/django/core/management/commands/migrate.py", line 114, in handle
executor = MigrationExecutor(connection,
self.migration_progress_callback)
^^^^^^^^^^^^^^^
File "/nix/store/2z5nsfr1b1hrk47as0sa3gxa2mdy2sck-
python3.14-django-6.0.4/lib/python3.14/site-
packages/django/db/migrations/executor.py", line 18, in __init__
self.loader = MigrationLoader(self.connection)
^^^^^^^^^^^^^^^
File "/nix/store/2z5nsfr1b1hrk47as0sa3gxa2mdy2sck-
python3.14-django-6.0.4/lib/python3.14/site-
packages/django/db/migrations/loader.py", line 59, in __init__
self.build_graph()
^^^^^^^^^^^
File "/nix/store/2z5nsfr1b1hrk47as0sa3gxa2mdy2sck-
python3.14-django-6.0.4/lib/python3.14/site-
packages/django/db/migrations/loader.py", line 282, in build_graph
self.load_disk()
^^^^^^^^^^^^^^^
File "/nix/store/2z5nsfr1b1hrk47as0sa3gxa2mdy2sck-
python3.14-django-6.0.4/lib/python3.14/site-
packages/django/db/migrations/loader.py", line 121, in load_disk
migration_module = import_module(migration_path)
^^^
File
"/nix/store/s33vdxkahhrdbyrqyw8yn71kzr44l6a0-python3-3.14.3/lib/python3.14/importlib/__init__.py",
line 88, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1398, in _gcd_import
File "<frozen importlib._bootstrap>", line 1371, in _find_and_load
File "<frozen importlib._bootstrap>", line 1335, in
_find_and_load_unlocked
ModuleNotFoundError: No module named
'tmpkv94tvgb.migrations.0001_initial_squashed_0002_auto_20260419_1921_squashed_0003_auto_20260419_1921'
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36997#comment:8>

[Django]

#36997: test_double_replaced_migrations_are_checked_correctly fails intermittently
under Windows Subsystem for Linux (WSL)
----------------------------------+--------------------------------------
Reporter: Stewart Matheson | Owner: (none)

Type: Bug | Status: new

Component: Migrations | Version: dev
Severity: Normal | Resolution:

Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+--------------------------------------

Changes (by mweinelt):

* resolution: needsinfo =>
* status: closed => new

--
Ticket URL: <https://code.djangoproject.com/ticket/36997#comment:9>

[Django]

#36997: test_double_replaced_migrations_are_checked_correctly fails intermittently
under Windows Subsystem for Linux (WSL)
----------------------------------+--------------------------------------
Reporter: Stewart Matheson | Owner: (none)

Type: Bug | Status: closed
Component: Migrations | Version: dev
Severity: Normal | Resolution: needsinfo

Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------+--------------------------------------

Changes (by Tim Graham):

* resolution: => needsinfo
* status: new => closed

Comment:

I don't see anything in the
[https://docs.djangoproject.com/en/6.0/releases/6.0.4/ Django 6.0.4
release notes] that looks relevant, so the fact that you started seeing
that after upgrading from 6.0.3 to 6.0.4 is probably a red herring. Is the
Python version relevant? I don't know. This is an obscure issue really
needs an explanation/confirmation of why Django is at fault before we can
do anything but triage the issue as "needsinfo".
--
Ticket URL: <https://code.djangoproject.com/ticket/36997#comment:10>