Unable to launch AWX UI using Load Balancer

[Kaushik R]

I'm unable to launch the AWX ui using http://<external-ip>:30102. I have the pods successfully deployed though -

NAME                                              TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)        AGE
awx-operator-awx-service                          LoadBalancer   10.100.197.40   xx.xx.xx.xx   80:30102/TCP   80m
awx-operator-controller-manager-metrics-service   ClusterIP      10.100.206.47   <none>          8443/TCP       81m

Any advise?

[AWX Project]

Hi, what errors are you getting / seeing when attempting to access that address?

AWX Team

[Kaushik R]

The url just keeps loading without any error for a long time and then gets timed out.

awx-web container logs:

[wait-for-migrations] Waiting for database migrations...

[wait-for-migrations] Attempt 1 of 30

[wait-for-migrations] Waiting 0.5 seconds before next attempt

[wait-for-migrations] Attempt 2 of 30

[wait-for-migrations] Waiting 1 seconds before next attempt

[wait-for-migrations] Attempt 3 of 30

[wait-for-migrations] Waiting 2 seconds before next attempt

[wait-for-migrations] Attempt 4 of 30

[wait-for-migrations] Waiting 4 seconds before next attempt

awx-task container logs:

[wait-for-migrations] Waiting for database migrations...

[wait-for-migrations] Attempt 1 of 30

[wait-for-migrations] Waiting 0.5 seconds before next attempt

[wait-for-migrations] Attempt 2 of 30

[wait-for-migrations] Waiting 1 seconds before next attempt

[wait-for-migrations] Attempt 3 of 30

[wait-for-migrations] Waiting 2 seconds before next attempt

[wait-for-migrations] Attempt 4 of 30

[wait-for-migrations] Waiting 4 seconds before next attempt

Redis container logs:

1:C 22 Jun 2022 15:06:49.121 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

1:C 22 Jun 2022 15:06:49.121 # Redis version=7.0.0, bits=64, commit=00000000, modified=0, pid=1, just started

1:C 22 Jun 2022 15:06:49.121 # Configuration loaded

1:M 22 Jun 2022 15:06:49.122 * monotonic clock: POSIX clock_gettime

1:M 22 Jun 2022 15:06:49.122 * Running mode=standalone, port=0.

1:M 22 Jun 2022 15:06:49.122 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.

1:M 22 Jun 2022 15:06:49.122 # Server initialized

1:M 22 Jun 2022 15:06:49.123 * The AOF directory appendonlydir doesn't exist

1:M 22 Jun 2022 15:06:49.123 * The server is now ready to accept connections at /var/run/redis/redis.sock

awx-ee container logs:

DEBUG 2022/07/11 18:02:22 Sending service advertisement: &{awx-7d5bd6b6d-rnp7b control 2022-07-11 18:02:22.849520469 +0000 UTC m=+1652132.737023817 1 map[type:Control Service] [{local false} {kubernetes-runtime-auth false} {kubernetes-incluster-auth false}]}
DEBUG 2022/07/11 18:03:22 Sending service advertisement: &{awx-7d5bd6b6d-rnp7b control 2022-07-11 18:03:22.852138837 +0000 UTC m=+1652192.739642185 1 map[type:Control Service] [{local false} {kubernetes-runtime-auth false} {kubernetes-incluster-auth false}]}
DEBUG 2022/07/11 18:04:22 Sending service advertisement: &{awx-7d5bd6b6d-rnp7b control 2022-07-11 18:04:22.854355392 +0000 UTC m=+1652252.741858745 1 map[type:Control Service] [{local false} {kubernetes-runtime-auth false} {kubernetes-incluster-auth false}]}

[AWX Project]

according to your logs, it looks like the migrations never ran to completion. It does take a while to run them; does the app eventually start after 30 minutes or so?

what is the status of your postgres pod? any error messages there?

AWX Team

[Kaushik R]

I'm using an external postgres instance. The database migration has already been completed.

Even then, the AWX containers still show this error of "Waiting for database migrations...". 

Is it because this AWX pod isn't able to access the externally hosted postgres instance?

[Manikandan Balasubramanian]

I have seen a similar issue. For me, it happened because the pod was not able to connect to Postgres DB. Can you please check whether pod is able to communicate with Postgres DB. If not, please check for your connectivity, firewall rules.

--
You received this message because you are subscribed to the Google Groups "AWX Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to awx-project...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/ceda372e-35fa-4542-af29-15d71c292bean%40googlegroups.com.

[Z M]

Can you advise how best to check if the pod is able to connect to the DB?

[Kaushik R]

I have checked the AWX pod connectivity to the external postgres instance and that works fine. Firewall has been whitelisted. Still get the database migrations pending logs...

[Z M]

This recently tripped me up when I was using an older version of AWX that was connecting to postgres11.

As soon as I started a newer instance running postgres12, this worked fine with the latest operator.

Can you confirm which version of a Db you're running?

Find the post that is not the pod/awx-operator, should pod/awx-*******-*****

Can you run the following commands:

kubectl -n yournamespace get awx,all,ingress,secrets
kubectl logs pod/awx-******* -n yournamespace -c redis
kubectl logs pod/awx-******* -n yournamespace -c awx-task
kubectl logs pod/awx-******* -n yournamespace -c awx-ee

To view this discussion on the web visit https://groups.google.com/d/msgid/awx-project/e0ab1edd-1dfa-4168-909c-7813892cfa47n%40googlegroups.com.

[Kaushik R]

I'm currently using awx-operator 0.23.0 and postgres14. I recently upgraded the postgres instance to v14. But was facing the same issue with v12 previously as well.

[user@hostname ~]$ kubectl -n ansible-awx get awx,all,ingress,secrets
NAME                      AGE
awx.awx.ansible.com/awx   6d1h

NAME                                     READY   STATUS    RESTARTS           AGE
pod/awx-7d5bd6b6d-xgfn2                  4/4     Running   1336 (6m19s ago)   6d1h
pod/controller-manager-dc95b975b-l4lzq   2/2     Running   0                  6d1h

NAME                                         TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)         AGE

service/awx-service                          LoadBalancer   10.100.199.20    x.x.x.x   443:30784/TCP   6d1h
service/controller-manager-metrics-service   ClusterIP      10.100.193.189   <none>          8443/TCP        6d1h

NAME                                 READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/awx                  1/1     1            1           6d1h
deployment.apps/controller-manager   1/1     1            1           6d1h

NAME                                           DESIRED   CURRENT   READY   AGE
replicaset.apps/awx-7d5bd6b6d                  1         1         1       6d1h
replicaset.apps/controller-manager-dc95b975b   1         1         1       6d1h

NAME                                    CLASS    HOSTS                   ADDRESS         PORTS   AGE
ingress.networking.k8s.io/awx-ingress   <none>   awx-dev.blackrock.com   x.x.x.x   80      6d1h

NAME                                    TYPE                                  DATA   AGE
secret/awx-admin-password               Opaque                                1      6d1h
secret/awx-app-credentials              Opaque                                3      6d1h
secret/awx-broadcast-websocket          Opaque                                1      6d1h
secret/awx-postgres-configuration       Opaque                                7      6d1h
secret/awx-secret-key                   Opaque                                1      6d1h
secret/awx-token-c4mbf                  kubernetes.io/service-account-token   3      6d1h
secret/controller-manager-token-4djz7   kubernetes.io/service-account-token   3      6d1h
secret/default-token-tbz2s              kubernetes.io/service-account-token   3      6d1h

I had posted all the container logs in my previous comments. 

[AWX Project]

| I'm using an external postgres instance. The database migration has already been completed.

How are you verifying that the migration is complete?

What does "awx-manage showmigrations" in the awx-task container show, or "awx-manage check_db" ?

Note that postgres 14 may work, but it is currently unsupported (we don't test it explicitly)

AWX Team

[Kaushik R]

Below are the awx-task container logs for "awx-manage showmigrations". I get the same error for check_db as well -

[user@hostname ~]$ kubectl exec awx-7d5bd6b6d-kj4c6 -c awx-task -n ansible-awx awx-manage showmigrations
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
2022-07-21 17:45:26,492 ERROR    [-] awx.main.utils.encryption Failed to decrypt `Setting(pk=20).value`; if you've recently restored from a database backup or are running in a clustered environment, check that your `SECRET_KEY` value is correct
Traceback (most recent call last):
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/cryptography/fernet.py", line 120, in _verify_signature
    h.verify(data[-32:])
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/cryptography/hazmat/primitives/hmac.py", line 69, in verify
    ctx.verify(signature)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/cryptography/hazmat/backends/openssl/hmac.py", line 76, in verify
    raise InvalidSignature("Signature did not match digest.")
cryptography.exceptions.InvalidSignature: Signature did not match digest.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/main/utils/encryption.py", line 159, in decrypt_field
    return smart_str(decrypt_value(key, value))
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/main/utils/encryption.py", line 136, in decrypt_value
    value = f.decrypt(encrypted)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/cryptography/fernet.py", line 81, in decrypt
    return self._decrypt_data(data, timestamp, time_info)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/cryptography/fernet.py", line 138, in _decrypt_data
    self._verify_signature(data)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/cryptography/fernet.py", line 122, in _verify_signature
    raise InvalidToken
cryptography.fernet.InvalidToken
Traceback (most recent call last):
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/cryptography/fernet.py", line 120, in _verify_signature
    h.verify(data[-32:])
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/cryptography/hazmat/primitives/hmac.py", line 69, in verify
    ctx.verify(signature)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/cryptography/hazmat/backends/openssl/hmac.py", line 76, in verify
    raise InvalidSignature("Signature did not match digest.")
cryptography.exceptions.InvalidSignature: Signature did not match digest.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/awx-manage", line 8, in <module>
    sys.exit(manage())
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/__init__.py", line 170, in manage
    execute_from_command_line(sys.argv)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/management/base.py", line 393, in execute
    self.check()
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/management/base.py", line 419, in check
    all_issues = checks.run_checks(
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/checks/registry.py", line 76, in run_checks
    new_errors = check(app_configs=app_configs, databases=databases)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/checks/urls.py", line 13, in check_url_config
    return check_resolver(resolver)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/checks/urls.py", line 23, in check_resolver
    return check_method()
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/urls/resolvers.py", line 416, in check
    for pattern in self.url_patterns:
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/utils/functional.py", line 48, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/urls/resolvers.py", line 602, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/utils/functional.py", line 48, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/urls/resolvers.py", line 595, in urlconf_module
    return import_module(self.urlconf_name)
  File "/usr/lib64/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/urls.py", line 12, in <module>
    re_path(r'^api/', include('awx.api.urls', namespace='api')),
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/urls/conf.py", line 34, in include
    urlconf_module = import_module(urlconf_module)
  File "/usr/lib64/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/api/urls/__init__.py", line 5, in <module>
    from .urls import urlpatterns
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/api/urls/urls.py", line 71, in <module>
    from .oauth2_root import urls as oauth2_root_urls
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/api/urls/oauth2_root.py", line 10, in <module>
    from oauth2_provider import views
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/oauth2_provider/views/__init__.py", line 2, in <module>
    from .base import AuthorizationView, TokenView, RevokeTokenView  # isort:skip
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/oauth2_provider/views/base.py", line 69, in <module>
    class AuthorizationView(BaseAuthorizationView, FormView):
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/oauth2_provider/views/base.py", line 93, in AuthorizationView
    server_class = oauth2_settings.OAUTH2_SERVER_CLASS
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/__init__.py", line 89, in oauth2_getattribute
    val = settings.OAUTH2_PROVIDER.get(attr)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/conf/settings.py", line 489, in __getattr_without_cache__
    return getattr(self._wrapped, name)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/cachetools/decorators.py", line 22, in wrapper
    v = func(*args, **kwargs)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/conf/settings.py", line 404, in __getattr__
    value = self._get_local(name)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/conf/settings.py", line 326, in _get_local
    self._preload_cache()
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/conf/settings.py", line 300, in _preload_cache
    value = decrypt_field(setting, 'value')
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/main/utils/encryption.py", line 159, in decrypt_field
    return smart_str(decrypt_value(key, value))
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/main/utils/encryption.py", line 136, in decrypt_value
    value = f.decrypt(encrypted)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/cryptography/fernet.py", line 81, in decrypt
    return self._decrypt_data(data, timestamp, time_info)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/cryptography/fernet.py", line 138, in _decrypt_data
    self._verify_signature(data)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/cryptography/fernet.py", line 122, in _verify_signature
    raise InvalidToken
cryptography.fernet.InvalidToken
command terminated with exit code 1

[AWX Project]

The error above suggests that your secret key is bad: `awx.main.utils.encryption Failed to decrypt `Setting(pk=20).value`; if you've recently restored from a database backup or are running in a clustered environment, check that your `SECRET_KEY` value is correct`.
Did you happen to point a new pod to an existing database without specifying the secret_key?
There are encrypted values stored in the database and the secret_key held by the pod is used to decrypt them. See: https://github.com/ansible/awx-operator#secret-key-configuration

There has been other threads on this group where others had this issue. i.e. https://groups.google.com/g/awx-project/c/qugKoMu4CHY/m/v3PnO_kDAgAJ

-The AWX Team

[Kaushik R]

Where can I get the value of this secret_key from in order to specify it in the AWX spec ? The database?

[AWX Project]

The key is not stored in the database.
It would be defined in the original name space you initially deployed AWX into as a secret.
Per the docs (https://github.com/ansible/awx-operator#secret-key-configuration) the default name would be "<resourcename>-secret-key".
Optionally it would also be in a backup created by the backup role (https://github.com/ansible/awx-operator/tree/devel/roles/backup).

-The AWX Team

[Kaushik R]

I have implemented the suggested changes for awx-secret-key in the spec and  secret_key.yaml.j2 but the execution still fails with the same error mentioned above.

Do I need to rebuild my database? Please note that the data stored in my database isn't encrypted. Do I still need this secret to decrypt the values stored in the DB?

[AWX Project]

According to this thread, if you are comfortable with DBs, you may be able to recover the non encrypted data  https://groups.google.com/g/awx-project/c/qugKoMu4CHY/m/v3PnO_kDAgAJ

Otherwise you'll need to rebuild the DB

AWX Team

[Kaushik R]

I was able to rebuild the database use the awx-secret-key in the spec. The error is resolved now, however I'm still not able to access the UI -

[user@hostname~]$ kubectl exec awx-7d5bd6b6d-pz9bd -c awx-task -n ansible-awx awx-manage check_db

kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.

Database Version: PostgreSQL 12.10 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44), 64-bit

[user@hostname~]$ kubectl exec awx-7d5bd6b6d-pz9bd -c awx-task -n ansible-awx awx-manage showmigrations

kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.

auth
 [X] 0001_initial
 [X] 0002_alter_permission_name_max_length
 [X] 0003_alter_user_email_max_length
 [X] 0004_alter_user_username_opts
 [X] 0005_alter_user_last_login_null
 [X] 0006_require_contenttypes_0002
 [X] 0007_alter_validators_add_error_messages
 [X] 0008_alter_user_username_max_length
 [X] 0009_alter_user_last_name_max_length
 [X] 0010_alter_group_name_max_length
 [X] 0011_update_proxy_permissions
 [X] 0012_alter_user_first_name_max_length
conf
 [X] 0001_initial
 [X] 0002_v310_copy_tower_settings
 [X] 0003_v310_JSONField_changes
 [X] 0004_v320_reencrypt
 [X] 0005_v330_rename_two_session_settings
 [X] 0006_v331_ldap_group_type
 [X] 0007_v380_rename_more_settings
 [X] 0008_subscriptions
 [X] 0009_rename_proot_settings
contenttypes
 [X] 0001_initial
 [X] 0002_remove_content_type_name
main
 [X] 0001_initial
 [X] 0002_squashed_v300_release (18 squashed migrations)
 [X] 0003_squashed_v300_v303_updates (9 squashed migrations)
 [X] 0004_squashed_v310_release (6 squashed migrations)
 [X] 0005_squashed_v310_v313_updates (3 squashed migrations)
 [X] 0006_v320_release
 [X] 0007_v320_data_migrations
 [X] 0008_v320_drop_v1_credential_fields
 [X] 0009_v322_add_setting_field_for_activity_stream
 [X] 0010_v322_add_ovirt4_tower_inventory
 [X] 0011_v322_encrypt_survey_passwords
 [X] 0012_v322_update_cred_types
 [X] 0013_v330_multi_credential
 [X] 0014_v330_saved_launchtime_configs
 [X] 0015_v330_blank_start_args
 [X] 0016_v330_non_blank_workflow
 [X] 0017_v330_move_deprecated_stdout
 [X] 0018_v330_add_additional_stdout_events
 [X] 0019_v330_custom_virtualenv
 [X] 0020_v330_instancegroup_policies
 [X] 0021_v330_declare_new_rbac_roles
 [X] 0022_v330_create_new_rbac_roles
 [X] 0023_v330_inventory_multicred
 [X] 0024_v330_create_user_session_membership
 [X] 0025_v330_add_oauth_activity_stream_registrar
 [X] 0026_v330_delete_authtoken
 [X] 0027_v330_emitted_events
 [X] 0028_v330_add_tower_verify
 [X] 0030_v330_modify_application
 [X] 0031_v330_encrypt_oauth2_secret
 [X] 0032_v330_polymorphic_delete
 [X] 0033_v330_oauth_help_text
 [X] 0034_v330_delete_user_role
 [X] 0035_v330_more_oauth2_help_text
 [X] 0036_v330_credtype_remove_become_methods
 [X] 0037_v330_remove_legacy_fact_cleanup
 [X] 0038_v330_add_deleted_activitystream_actor
 [X] 0039_v330_custom_venv_help_text
 [X] 0040_v330_unifiedjob_controller_node
 [X] 0041_v330_update_oauth_refreshtoken
 [X] 0042_v330_org_member_role_deparent
 [X] 0043_v330_oauth2accesstoken_modified
 [X] 0044_v330_add_inventory_update_inventory
 [X] 0045_v330_instance_managed_by_policy
 [X] 0046_v330_remove_client_credentials_grant
 [X] 0047_v330_activitystream_instance
 [X] 0048_v330_django_created_modified_by_model_name
 [X] 0049_v330_validate_instance_capacity_adjustment
 [X] 0050_v340_drop_celery_tables
 [X] 0051_v340_job_slicing
 [X] 0052_v340_remove_project_scm_delete_on_next_update
 [X] 0053_v340_workflow_inventory
 [X] 0054_v340_workflow_convergence
 [X] 0055_v340_add_grafana_notification
 [X] 0056_v350_custom_venv_history
 [X] 0057_v350_remove_become_method_type
 [X] 0058_v350_remove_limit_limit
 [X] 0059_v350_remove_adhoc_limit
 [X] 0060_v350_update_schedule_uniqueness_constraint
 [X] 0061_v350_track_native_credentialtype_source
 [X] 0062_v350_new_playbook_stats
 [X] 0063_v350_org_host_limits
 [X] 0064_v350_analytics_state
 [X] 0065_v350_index_job_status
 [X] 0066_v350_inventorysource_custom_virtualenv
 [X] 0067_v350_credential_plugins
 [X] 0068_v350_index_event_created
 [X] 0069_v350_generate_unique_install_uuid
 [X] 0070_v350_gce_instance_id
 [X] 0071_v350_remove_system_tracking
 [X] 0072_v350_deprecate_fields
 [X] 0073_v360_create_instance_group_m2m
 [X] 0074_v360_migrate_instance_group_relations
 [X] 0075_v360_remove_old_instance_group_relations
 [X] 0076_v360_add_new_instance_group_relations
 [X] 0077_v360_add_default_orderings
 [X] 0078_v360_clear_sessions_tokens_jt
 [X] 0079_v360_rm_implicit_oauth2_apps
 [X] 0080_v360_replace_job_origin
 [X] 0081_v360_notify_on_start
 [X] 0082_v360_webhook_http_method
 [X] 0083_v360_job_branch_override
 [X] 0084_v360_token_description
 [X] 0085_v360_add_notificationtemplate_messages
 [X] 0086_v360_workflow_approval
 [X] 0087_v360_update_credential_injector_help_text
 [X] 0088_v360_dashboard_optimizations
 [X] 0089_v360_new_job_event_types
 [X] 0090_v360_WFJT_prompts
 [X] 0091_v360_approval_node_notifications
 [X] 0092_v360_webhook_mixin
 [X] 0093_v360_personal_access_tokens
 [X] 0094_v360_webhook_mixin2
 [X] 0095_v360_increase_instance_version_length
 [X] 0096_v360_container_groups
 [X] 0097_v360_workflowapproval_approved_or_denied_by
 [X] 0098_v360_rename_cyberark_aim_credential_type
 [X] 0099_v361_license_cleanup
 [X] 0100_v370_projectupdate_job_tags
 [X] 0101_v370_generate_new_uuids_for_iso_nodes
 [X] 0102_v370_unifiedjob_canceled
 [X] 0103_v370_remove_computed_fields
 [X] 0104_v370_cleanup_old_scan_jts
 [X] 0105_v370_remove_jobevent_parent_and_hosts
 [X] 0106_v370_remove_inventory_groups_with_active_failures
 [X] 0107_v370_workflow_convergence_api_toggle
 [X] 0108_v370_unifiedjob_dependencies_processed
 [X] 0109_v370_job_template_organization_field
 [X] 0110_v370_instance_ip_address
 [X] 0111_v370_delete_channelgroup
 [X] 0112_v370_workflow_node_identifier
 [X] 0113_v370_event_bigint
 [X] 0114_v370_remove_deprecated_manual_inventory_sources
 [X] 0115_v370_schedule_set_null
 [X] 0116_v400_remove_hipchat_notifications
 [X] 0117_v400_remove_cloudforms_inventory
 [X] 0118_add_remote_archive_scm_type
 [X] 0119_inventory_plugins
 [X] 0120_galaxy_credentials
 [X] 0121_delete_toweranalyticsstate
 [X] 0122_really_remove_cloudforms_inventory
 [X] 0123_drop_hg_support
 [X] 0124_execution_environments
 [X] 0125_more_ee_modeling_changes
 [X] 0126_executionenvironment_container_options
 [X] 0127_reset_pod_spec_override
 [X] 0128_organiaztion_read_roles_ee_admin
 [X] 0129_unifiedjob_installed_collections
 [X] 0130_ee_polymorphic_set_null
 [X] 0131_undo_org_polymorphic_ee
 [X] 0132_instancegroup_is_container_group
 [X] 0133_centrify_vault_credtype
 [X] 0134_unifiedjob_ansible_version
 [X] 0135_schedule_sort_fallback_to_id
 [X] 0136_scm_track_submodules
 [X] 0137_custom_inventory_scripts_removal_data
 [X] 0138_custom_inventory_scripts_removal
 [X] 0139_isolated_removal
 [X] 0140_rename
 [X] 0141_remove_isolated_instances
 [X] 0142_update_ee_image_field_description
 [X] 0143_hostmetric
 [X] 0144_event_partitions
 [X] 0145_deregister_managed_ee_objs
 [X] 0146_add_insights_inventory
 [X] 0147_validate_ee_image_field
 [X] 0148_unifiedjob_receptor_unit_id
 [X] 0149_remove_inventory_insights_credential
 [X] 0150_rename_inv_sources_inv_updates
 [X] 0151_rename_managed_by_tower
 [X] 0152_instance_node_type
 [X] 0153_instance_last_seen
 [X] 0154_set_default_uuid
 [X] 0155_improved_health_check
 [X] 0156_capture_mesh_topology
 [X] 0157_inventory_labels
 [X] 0158_make_instance_cpu_decimal
 [X] 0159_deprecate_inventory_source_UoPU_field
oauth2_provider
 [X] 0001_initial
 [X] 0002_auto_20190406_1805
 [X] 0003_auto_20201211_1314
sessions
 [X] 0001_initial
sites
 [X] 0001_initial
 [X] 0002_alter_domain_unique
social_django
 [X] 0001_initial (2 squashed migrations)
 [X] 0002_add_related_name (2 squashed migrations)
 [X] 0003_alter_email_max_length (2 squashed migrations)
 [X] 0004_auto_20160423_0400 (2 squashed migrations)
 [X] 0005_auto_20160727_2333 (1 squashed migrations)
 [X] 0006_partial
 [X] 0007_code_timestamp
 [X] 0008_partial_timestamp
 [X] 0009_auto_20191118_0520
 [X] 0010_uid_db_index
sso
 [X] 0001_initial
 [X] 0002_expand_provider_options
taggit
 [X] 0001_initial
 [X] 0002_auto_20150616_2121
 [X] 0003_taggeditem_add_unique_index
 [X] 0004_alter_taggeditem_content_type_alter_taggeditem_tag

[Kaushik R]

I get the below error when trying to access AWX UI -

"The connection for this site is not secure"

[AWX Project]

That is a basic SSL security issue. Did you specify a cert during the installation? if not a self signed cert would have been generated and applied to the server. Most browsers have a way to accept a self signed cert. You should also be able to get a signed cert and tell the operator to install that cert see https://github.com/ansible/awx-operator#ingress-type 

- The AWX Team

[Kaushik R]

Do I access the AWX UI using the External IP mentioned in the awx-service or using the IPAddress mentioned in the awx-ingress?

NAME                                         TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)         AGE

service/awx-service                          LoadBalancer   x.x.x.x   x.x.x.x   443:31372/TCP   7d23h
service/controller-manager-metrics-service   ClusterIP      x.x.x.x   <none>          8443/TCP        7d23h

NAME          CLASS    HOSTS                   ADDRESS         PORTS   AGE

awx-ingress   <none>   awx-dev.com   x.x.x.x   80      7d23h

[Kaushik R]

The issue still persists even after applying self-signed SSL certs.

I have configured SSL certs as per guidelines under https://devopscube.com/configure-ingress-tls-kubernetes/.

I created the secret containing the certs before installing AWX. 

What could I be doing wrong?

[AWX Project]

Did you add your self signed CA cert to your browser's trust store, or your organization's certificate authority?

AWX Team