Problem psycopg2.errors.UndefinedFile: could not open file "base/16384/21268": No such file or directory

[송희령]

My AWX version is 21 and I installed it according to the guide on github. After a week, an error occurred when accessing the AWX account, and when I checked the log, it was as follows.

---------------------------------------

2022-06-27 08:00:39,511 ERROR [8c8014e8568a4071bdc2a9aa010a307e] django.request Internal Server Error: /api/login/
Traceback (most recent call last):
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
psycopg2.errors.UndefinedFile: could not open file "base/16384/21268": No such file or directory


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/lib64/python3.9/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/views/generic/base.py", line 70, in view
return self.dispatch(request, *args, **kwargs)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/utils/decorators.py", line 43, in _wrapper
return bound_method(*args, **kwargs)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/views/decorators/debug.py", line 89, in sensitive_post_parameters_wrapper
return view(request, *args, **kwargs)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/utils/decorators.py", line 43, in _wrapper
return bound_method(*args, **kwargs)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
response = view_func(request, *args, **kwargs)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/utils/decorators.py", line 43, in _wrapper
return bound_method(*args, **kwargs)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/views/decorators/cache.py", line 44, in _wrapped_view_func
response = view_func(request, *args, **kwargs)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/contrib/auth/views.py", line 63, in dispatch
return super().dispatch(request, *args, **kwargs)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/views/generic/base.py", line 98, in dispatch
return handler(request, *args, **kwargs)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/api/generics.py", line 99, in post
current_user = smart_str(JSONRenderer().render(current_user.data))
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/rest_framework/renderers.py", line 100, in render
ret = json.dumps(
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/rest_framework/utils/json.py", line 25, in dumps
return json.dumps(*args, **kwargs)
File "/usr/lib64/python3.9/json/__init__.py", line 234, in dumps
return cls(
File "/usr/lib64/python3.9/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib64/python3.9/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/rest_framework/utils/encoders.py", line 47, in default
return tuple(obj)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/db/models/query.py", line 280, in __iter__
self._fetch_all()
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/db/models/query.py", line 1324, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/db/models/query.py", line 109, in __iter__
for row in compiler.results_iter(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size):
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/db/models/sql/compiler.py", line 1130, in results_iter
results = self.execute_sql(MULTI, chunked_fetch=chunked_fetch, chunk_size=chunk_size)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql
cursor.execute(sql, params)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
django.db.utils.OperationalError: could not open file "base/16384/21268": No such file or directory

------------------------------

Can you tell me what went wrong and what is the solution?

[AWX Project]

We haven't seen this before. Once it gets into this state, do you continue to hit this error? Are you doing anything special with your postgres setup (external)?

Could you provide more info about your setup? Thanks!

AWX Team

[송희령]

No, I don't use external postgres, and never installed any version of postgres.

I just Installed according to the guidelines(https://github.com/ansible/awx/blob/devel/INSTALL.md).

After a week the install AWX always cannot login AWX with these errors.

I load this at minikube, not k8s. OS is Linux CentOS 7.

2022년 6월 30일 목요일 오전 4시 31분 51초 UTC+9에 AWX Project님이 작성:

[AWX Project]

Interesting, do the postgres pod logs report any errors / messages while trying to log in?

AWX Team